Reconfigurable parallel processor with a plurality of chained memory ports

ABSTRACT

Processors, systems and methods are provided for thread level parallel processing. A processor may comprise a plurality of processing elements (PEs) that each may comprise a configuration buffer, a sequencer coupled to the configuration buffer of each of the plurality of PEs and configured to distribute one or more PE configurations to the plurality of PEs, and a gasket memory coupled to the plurality of PEs and being configured to store at least one PE execution result to be used by at least one of the plurality of PEs during a next PE configuration.

RELATED APPLICATION FIELD

The application claims priority to U.S. Provisional Application62/471,340, filed Mar. 14, 2017, entitled “Reconfigurable ParallelProcessing,” U.S. Provisional Application 62/471,367, filed Mar. 15,2017, entitled “Circular Reconfiguration for Reconfigurable ParallelProcessor,” U.S. Provisional Application 62/471,368, filed Mar. 15,2017, entitled “Private Memory Structure for Reconfigurable ParallelProcessor,” U.S. Provisional Application 62/471,372, filed Mar. 15,2017, entitled “Shared Memory Structure for Reconfigurable ParallelProcessor,” U.S. Provisional Application 62/472,579, filed Mar. 17,2017, entitled “Static Shared Memory Access for Reconfigurable ParallelProcessor,” the contents of these applications are hereby incorporatedby reference in their entities.

TECHNICAL FIELD

The disclosure herein relates to computer architectures, particularlyrelates to reconfigurable processors.

BACKGROUND

Reconfigurable computing architecture with large amount of processingarray can meet demand of computation power while keeping the power andsilicon area efficient. Unlike field-programmable gate array (FPGA),Coarse-Grained Reconfigurable Architecture (CGRA) utilizes largerprocessing elements like arithmetic logic units (ALU) as its buildingblocks. It provides features of reconfigurability using high levellanguage to quickly program the processing element (PE) array. Onetypical design of CGRA is shown in FIG. 1. It consists of a PE array, aconfiguration memory, a memory unit as a frame buffer andinterconnections between the PEs and PEs to the frame buffer.

In general, CGRA is an approach to explore loop level parallelism. It isnot specifically targeted to handle thread level parallelism. With anydata dependency from one iteration to the next, the parallelism islargely limited. Therefore, the 2D array size is intended to be limitedto an 8×8 PE array in most of the designs.

Graphics processing unit (GPU) architecture has provided a way toexecute parallel threads in a Same Instruction Multiple Thread (SIMT)fashion. It is especially suitable for massive parallel computingapplications. In these applications, typically no dependency is assumedbetween threads. This type of parallelism is beyond loop levelparallelism within a software task, which CGRA is designed for. Thethread level parallelism can be easily scalable beyond single coreexecution to multicore execution. The thread parallelism providesoptimization opportunities and makes the PE array more efficient andmore capable and it is easily made larger than 8×8. GPU, however, is notreconfigurable. Therefore, there is a need in the art to develop a nextgeneration processor that can harness the processing capability of bothCGRA and GPU.

SUMMARY

The present disclosure describes apparatus, methods and systems formassive parallel data processing. A processor according to variousembodiments of the present disclosure may be designed to take advantageof massive thread level parallelism similar to GPU using programableprocessor array similar to CGRA. In one embodiment, a processor mayefficiently process threads which are identical to each other but withdifferent data, similar to SIMT architecture. A software program's datadependency graph may be mapped to a virtual data path with infinitelength. Then the virtual data path may be chopped into segments that canbe fit into multiple physical data paths, each physical data path mayhave its configuration context. A sequencer may distributeconfigurations of each PE into its configuration FIFO and similarly toswitch boxes. A gasket memory may be used to temporarily store outputsof one physical data path configuration and give it back to theprocessing elements for the next configuration. Memory ports may be usedto calculate addresses for read and write. FIFOs may be used to alloweach PE for independent operation. Data stored in a memory unit may beaccessed through either private or shared memory access method. The samedata can be accessed through different access methods in different partof the software program to reduce data movement between memories.

In an exemplary embodiment, there is provided a processor comprising: aplurality of processing elements (PEs) each comprising a configurationbuffer; a sequencer coupled to the configuration buffer of each of theplurality of PEs and configured to distribute one or more PEconfigurations to the plurality of PEs; and a gasket memory coupled tothe plurality of PEs and being configured to store at least one PEexecution result to be used by at least one of the plurality of PEsduring a next PE configuration.

According to an embodiment, the processor may further comprise aplurality of switch boxes coupled to the sequencer to receive switch boxconfigurations from the sequencer, each of the plurality of switch boxesmay be associated with a respective PE of the plurality of PEs andconfigured to provide input data switching for the respective PEaccording to the switch box configurations.

According to an embodiment, the plurality of switch boxes and theirassociated PEs are arranged in a plurality of columns, a first switchbox in a first column of the plurality of columns is coupled between thegasket memory and a first PE in the first column of the plurality ofcolumns, and a second PE in a last column of the plurality of columns iscoupled to the gasket memory.

According to an embodiment, the processor may further comprise a memoryunit for providing data storage for the plurality of PEs; and aplurality of memory ports each arranged in a separate column of theplurality of columns for the plurality of PEs to access the memory unit.

According to an embodiment, the processor may further comprise aplurality of inter-column switch boxes (ICSBs) coupled to the sequencerto receive ICSB configurations from the sequencer, the plurality ofICSBs being configured to provide data switching between neighboringcolumns of the plurality of columns according to the ICSBconfigurations.

According to an embodiment, the plurality of memory ports (MPs) arecoupled to the sequencer to receive MP configurations from the sequencerand configured to operate in a private access mode or a shared accessmode during one MP configuration.

According to an embodiment, a piece of data stored in the memory unit isaccessed through the private access mode and the shared access mode indifferent part of a program without the piece of data being moved in thememory unit.

According to an embodiment, each of the plurality of columns comprisesone PE, the plurality of PEs are identical and form one row ofrepetitive identical PEs.

According to an embodiment, each of the plurality of columns comprisestwo or more PEs and the plurality of PEs form two or more rows.

According to an embodiment, a first row of PEs are configured toimplement a first set of instructions and a second row of PEs areconfigured to implement a second set of instructions, at least oneinstruction of the second set of instructions is not in the first set ofinstructions, wherein the of plurality of columns are identical and formrepetitive columns.

According to an embodiment, each of the plurality of PEs comprises aplurality of arithmetic logic units (ALUs) that are configured toexecute same instruction in parallel threads.

According to an embodiment, each of the plurality of PEs comprises aplurality of data buffers for the plurality of ALUs and is configured tooperate independently.

According to an embodiment, each of the plurality of memory ports isconfigured to access the memory unit using a vector address, wherein inthe private access mode, one address in the vector address is routed toone memory bank of the memory unit according to a thread index and allprivate data for one thread are located in a same memory bank.

According to an embodiment, each of the plurality of memory ports isconfigured to access the memory unit using a vector address, wherein inthe shared access mode, one address in the vector address is routed in adefined region across memory banks regardless of the thread index anddata shared to all threads are spread in all memory banks.

In another exemplary embodiment, there is provided a method comprising:mapping an execution kernel into a virtual data path at a processor,wherein the execution kernel includes a sequence of instructions to beexecuted by the processor, and the processor comprises variousreconfigurable units that include a gasket memory; chopping the virtualdata path into one or more physical data paths; deliveringconfigurations to various reconfigurable units of the processor for thevarious reconfigurable units to form the one or more physical data pathsto execute the sequence of instructions; and executing the processor tocomplete the one or more physical data paths by operating the variousreconfigurable units according to the configurations, including routingdata from one physical data path to the gasket memory to be used in afuture physical data path as input.

According to an embodiment, the various reconfigurable units furthercomprise a plurality of processing elements, a plurality of switch boxeseach associated with a separate processing element, a plurality ofmemory ports that provide access to a memory unit for the plurality ofprocessing elements, and a plurality of inter-column switch boxes, whereeach of the various reconfigurable units are reconfigured by applying anext configuration independently from other reconfigurable units.

According to an embodiment, each of the plurality of PEs comprises aplurality of arithmetic logic units (ALUs) that are configured toexecute same instruction in parallel threads.

According to an embodiment, each of the plurality of memory ports isconfigured to operate in a private access mode or a shared access modeduring one configuration.

According to an embodiment, the method may further comprise accessing apiece of data stored in the memory unit through the private access modeand the shared access mode in different physical data paths without thepiece of data being moved in the memory unit.

According to an embodiment, each of the memory ports is configured toaccess the memory unit using a vector address, wherein in the privateaccess mode, one address in the vector address is routed to one memorybank of the memory unit according to a thread index and all private datafor one thread are located in a same memory bank, and wherein in theshared access mode, one address in the vector address is routed in adefined region across memory banks regardless of the thread index anddata shared to all threads are spread in all memory banks.

According to an embodiment, each of the plurality of PEs comprises aplurality of data buffers for the plurality of ALUs and is configured tooperate independently during one physical data path.

According to an embodiment, the plurality of PEs form a PE array and theexecution kernel is mapped into one or more physical data paths on theprocessor based on a size of the PE array, connections between theplurality of PEs, and memory access capability.

According to an embodiment, the one or more physical data paths have asame repetitive structure that are made of the various reconfigurableunits.

In yet another exemplary embodiment, there is provided a systemcomprising: a processor, comprising: a sequencer configured to map anexecution kernel to be executed by the processor into a virtual datapath and chop the virtual data path into one or more physical datapaths; a plurality of processing elements (PEs) coupled to thesequencer, each of the plurality of PEs comprising a configurationbuffer configured to receive PE configurations for the one or morephysical data paths from the sequencer; and a gasket memory coupled tothe plurality of PEs and being configured to store data from one of theone or more physical data paths to be used by another physical data pathof the one or more physical data paths as input.

According to an embodiment, the processor further comprises a pluralityof switch boxes (SBs) coupled to the sequencer to receive SBconfigurations for the one or more physical data paths from thesequencer, each of the plurality of switch boxes being associated with arespective PE of the plurality of PEs and configured to provide inputdata switching for the respective PE according to the SB configurations.

According to an embodiment, the plurality of switch boxes and theirassociated PEs are arranged in a plurality of columns, a first switchbox in a first column of the plurality of columns is coupled between thegasket memory and a first PE in the first column of the plurality ofcolumns, and a second PE in a last column of the plurality of columns iscoupled to the gasket memory.

According to an embodiment, the processor further comprises: a memoryunit for providing data storage for the plurality of PEs; and aplurality of memory ports each arranged in a separate column of theplurality of columns for the plurality of PEs to access the memory unit.

According to an embodiment, the processor further comprises a pluralityof inter-column switch boxes (ICSBs) coupled to the sequencer to receiveICSB configurations for the one or more physical data paths from thesequencer, the plurality of ICSBs being configured to provide dataswitching between neighboring columns of the plurality of columnsaccording to the ICSB configurations.

According to an embodiment, the plurality of memory ports (MPs) arecoupled to the sequencer to receive MP configurations for the one ormore physical data paths from the sequencer and configured to operate ina private access mode or a shared access mode during one MPconfiguration.

According to an embodiment, a piece of data stored in the memory unit isaccessed through the private access mode and the shared access mode indifferent physical data paths of the one or more physical data pathswithout the piece of data being moved in the memory unit.

According to an embodiment, each of the plurality of columns comprisesone PE and the plurality of PEs are identical and form one row ofrepetitive identical PEs.

According to an embodiment, each of the plurality of columns comprisestwo or more PEs and the plurality of PEs form two or more rows.

According to an embodiment, a first row of PEs are configured toimplement a first set of instructions and a second row of PEs areconfigured to implement a second set of instructions, at least oneinstruction of the second set of instructions is not in the first set ofinstructions, wherein the of plurality of columns are identical and formrepetitive columns.

According to an embodiment, each of the plurality of PEs comprises aplurality of arithmetic logic units (ALUs) that are configured toexecute same instruction in parallel threads.

According to an embodiment, each of the plurality of PEs comprises aplurality of data buffers for the plurality of ALUs and is configured tooperate independently.

According to an embodiment, each of the plurality of memory ports isconfigured to access the memory unit using a vector address, wherein inthe private access mode, one address in the vector address is routed toone memory bank of the memory unit according to a thread index and allprivate data for one thread are located in a same memory bank.

According to an embodiment, each of the plurality of memory ports isconfigured to access the memory unit using a vector address, wherein inthe shared access mode, one address in the vector address is routed in adefined region across memory banks regardless of the thread index anddata shared to all threads are spread in all memory banks.

According to an embodiment, the plurality of PEs form a PE array and theexecution kernel is mapped into one or more physical data paths on theprocessor based on a size of the PE array, connections between theplurality of PEs, and memory access capability.

In yet another exemplary embodiment, there is provided a processor,comprising: a plurality of processing elements (PEs); a plurality ofswitch boxes arranged in a plurality of columns, each of the pluralityof switch boxes being associated with a respective PE and configured toprovide input data switching for the respective PE; a plurality ofmemory ports arranged in the plurality of columns and being coupled to amemory unit and a top switch box in each column of the plurality ofcolumns, each of the plurality of memory port being configured toprovide data access to the memory unit for one or more switch boxes in arespective column; a plurality of inter-column switch boxes (ICSBs) eachcoupled to a bottom switch box in each column of the plurality ofcolumns; and a gasket memory with its input coupled to a memory port, aPE, one or more switch boxes and an ICSB in a last column of theplurality of columns, and its output coupled to a memory port, one ormore switch boxes and an ICSB in a first column of the plurality ofcolumns.

According to an embodiment, the processor may further comprise asequencer coupled to the plurality of PEs, the plurality of switchboxes, the plurality of ICSBs, the plurality of memory ports and thegasket memory to deliver configurations to these components.

According to an embodiment, the processor may further comprise aconfiguration memory coupled to the sequencer to store compiledconfigurations for the sequencer to decode and deliver.

According to an embodiment, the processor may further comprise a memoryunit for providing data storage for the processor.

BRIEF DESCRIPTION OF FIGURES

FIG. 1 schematically shows a prior art CGRA with a PE array.

FIG. 2 schematically shows a processor in accordance with an embodimentof the present disclosure.

FIG. 3A schematically shows a memory system for a processor inaccordance with an embodiment of the present disclosure.

FIG. 3B schematically shows a first memory mapping for a private memoryaccess mode in accordance with an embodiment of the present disclosure.

FIG. 3C schematically shows a second memory mapping for a private memoryaccess mode in accordance with an embodiment of the present disclosure.

FIG. 3D schematically shows a memory mapping for a shared memory accessin accordance with an embodiment of the present disclosure.

FIG. 4A schematically shows a first memory access configuration for amemory port in accordance with an embodiment of the present disclosure.

FIG. 4B schematically shows a second memory access configuration for amemory port in accordance with an embodiment of the present disclosure.

FIG. 5 schematically shows a switch box for a processor in accordancewith an embodiment of the present disclosure.

FIG. 6A schematically shows a processing element for a processor inaccordance with an embodiment of the present disclosure.

FIG. 6B schematically shows another processing element for a processorin accordance with an embodiment of the present disclosure.

FIG. 7 schematically shows an inter-column switch box for a processor inaccordance with an embodiment of the present disclosure.

FIG. 8 schematically shows a gasket memory for a processor in accordancewith an embodiment of the present disclosure.

FIG. 9A schematically shows a dependency graph of an execution kernel inaccordance with an embodiment of the present disclosure.

FIG. 9B schematically shows the dependency graph of the execution kernelof FIG. 9A mapped into a virtual data path for a processor in accordancewith an embodiment of the present disclosure.

FIG. 9C schematically shows the virtual data path of FIG. 9B dividedinto physical data paths for a processor in accordance with anembodiment of the present disclosure.

FIG. 10 schematically shows a pipeline operation for a processor inaccordance with an embodiment of the present disclosure.

FIGS. 11A-11I schematically shows a configuration process for aprocessor in accordance with an embodiment of the present disclosure.

FIG. 12A schematically shows a pipeline operation for executing a streamof instructions using a processor in accordance with an embodiment ofthe present disclosure.

FIG. 12B schematically shows a reduced pipeline operation for executinga stream of instructions using a processor in accordance with anembodiment of the present disclosure.

FIG. 13 is a flowchart of a method for executing an execution kernel inaccordance with an embodiment of the present disclosure.

FIG. 14 is a flowchart of a method for reconfiguration in accordancewith an embodiment of the present disclosure.

FIG. 15 is a flowchart of another method for reconfiguration inaccordance with an embodiment of the present disclosure.

FIG. 16 is a flowchart of another method for reconfiguration inaccordance with an embodiment of the present disclosure.

FIG. 17 is a flowchart of a method for accessing memory in accordancewith an embodiment of the present disclosure.

FIG. 18 is a flowchart of another method for accessing memory inaccordance with an embodiment of the present disclosure.

FIG. 19 is a flowchart of a method for reusing a piece of data inaccordance with an embodiment of the present disclosure.

DETAILED DESCRIPTION

Reference will now be made in detail to the embodiments of the presentteaching, examples of which are illustrated in the accompanyingdrawings. Like elements in the various figures are denoted by likereference numerals for consistency. While the present teaching will bedescribed in conjunction with the embodiments, it will be understoodthat they are not intended to limit the present teaching to theseembodiments. On the contrary, the present teaching is intended to coveralternatives, modifications, and equivalents, which may be includedwithin the spirit and scope of the present teaching as defined by theappended claims.

In addition, in the following detailed description of embodiments of thepresent teaching, numerous specific details are set forth in order toprovide a thorough understanding of the present teaching. However, itwill be recognized by one of ordinary skill in the art that the presentteaching may be practiced without these specific details. In otherinstances, well-known methods, procedures, components, and circuits havenot been described in detail as not to unnecessarily obscure aspects ofthe embodiments of the present teaching.

FIG. 2 schematically shows a processor 200 according to an embodiment ofa processor. The processor 200 may comprise a direct memory access (DMA)module 202, a configuration memory 204, a sequencer 206, a controlprocessor 208, a program cache 210, a memory unit 212, a PE array 214and a gasket memory 216. The DMA module 202 may be coupled to anexternal bus 230 and may be controlled by the control processor 208. TheDMA module 202 may be responsible to move executable instructions andnon-executable data in and out from the external bus 230. The programcache 210 may store instructions and data to be used by the controlprocessor 208 to control the operation of the DMA module 202. In oneembodiment, the instructions and data stored in the program cache 210may be used by the control processor 208 to handle sequencer programs.

It should be noted that as used herein, a “coupling” between twocomponents, such as one component being “coupled” to another componentmay refer to an electronic connection between two components, which mayinclude but not limited to, by electronic wiring, through an electronicelement (e.g., a resistor, a transistor), etc. Moreover, in someembodiments, the processor 200 may be configured for massive threadlevel parallel processing. For example, one processing element (PE) inthe PE array 214 may comprise multiple arithmetic logic units (ALUs)that may be configured to perform same operation but on different data(e.g., each in a separate thread). That is, in these embodiments withmultiple ALUs, each PE may be configured to operate in a SingleInstruction Multiple Threads (SIMT) fashion. In one embodiment, a PEwith a vector address and a vector data input may generate vector dataoutput. In some embodiments, a thread may also be referred to as astream.

To provide data for multiple threads to be executed concurrently, insome embodiments, some relevant electronic connections betweencomponents of the processor 200 may be in vectors. For example, a vectoraddress of H×G may have H number of G bits addresses, a vector dataconnection of K×W may have K number of W bits data. It should also benoted that although not shown in any of the figures, data or addressconnections between different components may be accompanied by one ormore signal lines. For example, a busy signal line may exist between afirst component and a second component, and may be used by the firstcomponent to send a busy signal to the second component indicating thatthe first component is not ready to accept valid data or addresssignals. Moreover, a valid signal line may also exist between the firstand second components, and may be used by the second component to send avalid signal to the first component indicating that valid data oraddress signals have been put on the connection wires.

The configuration memory 204 may store data path programs consisting ofexecutable instructions and/or data loading instructions for one or moredata paths. In one embodiment, the data path programs stored in theconfiguration memory 204 may be sequence(s) of compiled instructions.For example, a data path program may include instructions to be executedby the PE array 214, which represents configuration information to beexecuted by which PEs when conditions are met, and how each data pathcomponents may hold or transmit data.

The sequencer 206 may decode the instruction stored in the configurationmemory 204 and move a decoded instruction into the memory unit 212 and aphysical data path. The physical data path may include variouscomponents of the PE array 214 (e.g., components of the PE array 214that will be involved in the executing on, staging and/or movement ofdata) and the gasket memory 216. The decoded instruction may bedelivered to various components in a package, which may be referred toas a configuration package or simply a configuration. In addition to thedecoded instruction, a configuration package for one component mayinclude some other parameters (e.g., a number specifying how many timesan instruction is to be repeatedly executed or how many times datapasses through a data switching unit in one configuration setting). Inone embodiment, a physical data path configuration may be referred to asa physical data path program, which may comprise individualconfigurations for various components included in a physical data path.Although not shown, there may be a configuration bus connecting thesequencer 206 to the components of a data path for individualconfigurations to be delivered to these components via the busrespectively.

The memory unit 212 may be a data staging area to store data receivedfrom the external bus 230 and store execution result data generated bythe PE array 214 (before these results may be transmitted away via theexternal bus 230). In some embodiments, the memory unit 212 may be an inprocessor cache for a large memory system external of the processor 200.The PE array 214 may comprise a plurality of memory ports (MPs)220.1-220.N, a plurality of switch boxes (SBs) 222.1-222.N, a pluralityof processing elements (PEs) 218.1-218.N and a plurality of inter-columnswitch boxes (ICSBs) 224.1-224.N. In the example shown in FIG. 2, the PEarray 214 may comprise one row of processing elements 218.1-218.N (e.g.,one dimensional array), which may include N PEs with N being an integernumber. In the embodiments described herein, the number N may be 32. Butthis is an example and it may be another integer number in otherembodiments, such as, but not limited to, 16, 64, etc. Although theseexample numbers given may be power of 2, the number of PEs in one rowneed not be a power of 2 in one embodiment. In some embodiments, the PEarray 214 may be a two-dimensional array that may comprise multiple rowsof processing elements (e.g., one or more rows of PEs may be positionedunderneath the PEs 218.1-218.N). It should be noted that the PE array214 may be a composite of MPs, SBs, ICSBs and PEs for illustrationpurpose and used to refer to these components collectively. In someembodiments, the memory unit 212 and the MPs 220.1-220.N may becollectively referred to collectively as a multi-port memory unit.

The plurality of MPs 220.1-220.N may be gateways for data flow betweenthe PE array 214 and the memory unit 212. Each MP 220.1-220.N may becoupled to the memory unit 212 respectively to read from and write tothe memory unit 212. With the exception of MP 220.1 and MP 220.N, allMPs may be coupled to two adjacent MPs such that each MP may beconfigured to receive data from a first MP and/or transmit data to asecond MP. The electronic coupling between MPs may provide a one-wayflow of data (e.g., if one computation configuration specifies that datamay flow from one MP to a next MP). For example, as shown in FIG. 2, MP220.1 may be coupled to MP 220.2 for one-way flow of data, MP 220.2 maybe coupled to MP 220.3 for one-way flow of data. The last MP 220.N maybe an exception and coupled to the gasket memory 216, which may providea temporary storage for data. The first MP 220.1 may be anotherexception in that it may receive one-way flow of data from the gasketmemory 216. In some embodiments, the MPs 220.1-220.N may form a datarouting bus along a PE row direction. That is, data may be routedbetween MPs in a direction that is parallel to the direction that datamay be routed between PEs. In embodiments with two-dimensional PE array214, each MP 220.1-220.N may be shared by one column of PEs. In oneembodiment, the gasket memory 216 may be used as a data buffer, forexample, first-in-first-out (FIFO), to collect data from the PE arrayand feed it back to the PE array for a new configuration.

In some embodiments, the PEs and MPs may be statically programmed withinstructions for one configuration. For example, the instructions may beprogrammed to the PEs and MPs as a stage of pipeline and no instructionsare changed during one configuration. The address computationinstructions and memory access instructions like read or store may bemapped to the memory ports (MP) and other instructions may be mapped toPEs.

As illustrated in FIG. 2, each of the ICSB 224.1-224.N may be coupled toa respective SB 222.1-222.N and two adjacent ICSBs, with the exceptionof ICSB 224.1 and ICSB 224.N. ICSB 224.1 may be coupled to the MP 222.1,the gasket memory 216 and the ICSB 224.2. And ICSB 224.N may be coupledto the MP 222.N, the gasket memory 216 and the ICSB 224.N−1. In someembodiments, the ICSBs 220.1-220.N may form another data routing busalong a PE row direction. That is, data may be routed between ICSBs in adirection that is parallel to the direction that data may be routedbetween PEs. An ICSB may also be referred to as a data switching unit.In one embodiment, one or more ICSBs may be used to route data to bypassone or more PEs.

The SBs 222.1-222.N may be configured to provide data switching forneighboring PEs, PEs to data routing buses, and the data routing busformed by the MPs 220.1-220.N and the data routing bus formed by theICSBs 224.1-224.N. For example, the switch box 222.1 may be configuredto provide data switching for data to be delivered to the processingelement 218.1 from the gasket memory 216, the MP 220.1 and the ICSB224.1. Moreover, the switch box 222.1 may be configured to route databetween the gasket memory 216, the MP 220.1 and the ICSB 224.1. Asanother example, the switch box 222.2 may be configured to provide dataswitching for data to be delivered to the processing element 218.2 fromthe processing element 218.1, the MP 220.2 and the ICSB 224.2. Moreover,the switch box 222.2 may be configured to route data between theprocessing element 218.2, the MP 220.2 and the ICSB 224.2. In yetanother example, the switch box 222.N may be configured to provide dataswitching for data to be delivered to the processing element 218.N fromthe PE 218.N−1, the MP 220.N and the ICSB 224.N. Moreover, the switchbox 222.N may be configured to route data between PE 218.N−1, MP 220.Nand ICSB 224.N. A SB may also be referred to as a data switching unit.

An exemplary data path may be illustrated by the exemplary internalconnections of the SBs 222.1 to 222.N. For example, as shown in FIG. 2,SB 222.1 may show that two inputs of PE 218.1 may be coupled to twooutputs from MP 220.1, SB 222.2 may show that two inputs of PE 218.2 maybe coupled to two outputs from MP 220.2 and two inputs of PE 218.2 maybe coupled to two outputs from PE 218.1, SB 222.3 may show that twoinputs of PE 218.3 may be coupled to two outputs from MP 220.3 and twoinputs of PE 218.3 may be coupled to two outputs from PE 218.2, and soforth, until SB 222.N may show that two inputs of PE 218.N may becoupled to two outputs from MP 220.N and two inputs of PE 218.N may becoupled to two outputs from PE 218.N−1.

To simplify wording, a MP 220 may refer to one of the MPs 220.1-220.N, aSB 222 may refer to one of the SBs 222.1-222.N, a PE 218 may refer toone of the PEs 218.1-218.N and an ICSB 224 may refer to one of the ICSB224.1-224.N.

FIG. 3A schematically shows a memory system for a processor inaccordance with an embodiment of the present disclosure. The memorysystem may comprise a memory unit 300 and the plurality of memory ports220.1 through 220.N. The memory unit 300 may be an embodiment of thememory unit 212 in FIG. 2, and may comprise a plurality of memory banks(e.g., memory bank 0 denoted as 302.1, memory bank 1 denoted as 302.N,memory bank N−1 denoted as 302.N, etc.) and a plurality of memory caches304.1 through 304.N. Each of the memory bank 302 may be coupled to arespective cache 304. For example, the memory bank 302.1 may be coupledto the cache 304.1, the memory bank 302.2 may be coupled to the cache304.2, the memory bank 302.N may be coupled to the cache 304.N, and soon. Each cache 304 may be individually coupled to all of the pluralityof memory ports 220.1-220.N. For example, the cache 304.1 may be coupledto the MPs 220.1-220.N, the cache 304.2 may be coupled to the MPs220.1-220.N, the cache 304.N may be coupled to the MPs 220.1-220.N, andso on.

In addition to being individually coupled to all caches 304.1-304.N ofthe memory unit 300, the MPs 220.1-220.N may be chained to form the rowdirection data routing bus, with the MP 220.1 and the MP 220.N beingcoupled at one end to the gasket memory 216 respectively (shown in FIG.2). Each of MPs 220.1-220.N may further comprise a write data (WData)input 306 and a read data (RData) output 308. For example, the MP 220.1may comprise a write data input 306.1 and a read data output 308.1, theMP 220.2 may comprise a write data input 306.2 and a read data output308.2, the MP 220.N may comprise a write data input 306.N and a readdata output 308.N, and so on. The write data inputs 306 and read dataoutputs 308 may be coupled to respective outputs and inputs of the SBs222.1-222.N. In one embodiment, each of the write data inputs 306 andeach of the read data outputs 308 may be configured for vector dataconnections. For example, the write data input 306.1 may be a 32×32input or two 32×16 inputs, and the read data output 308.1 may be a 32×32output or two 32×16 outputs. As used herein, a data input or data outputmay also be referred to a data port.

The memory unit 300 and MPs 220.1-220.N may support two accessing modes:a private memory access mode and a shared memory access mode, which mayalso be referred to as the private memory access method and sharedmemory access method. In one MP, multiple data units may be read orwritten using a vector address. These addresses of one vector may bedifferent from each other. In the private memory access mode, oneaddress in a vector address may be routed to one memory bank accordingto the thread index. All private data for one thread may be located inthe same memory bank. In shared memory access mode, each MP may accessanywhere in the defined region regardless of thread index. Data sharedto all threads may be spread in all memory banks.

The memory unit structure is illustrated as one example. For each columnof a PE array, it may have one MP with multiple buses going through. Thememory port may be configured as shared (e.g., shared memory accessmode) or private (e.g., private memory access mode). Each memory portmay be further coupled to a data cache network.

FIG. 3B schematically shows a first memory mapping for a private memoryaccess mode in accordance with an embodiment of the present disclosure.Each memory bank 302.1 through 302.N may comprise a plurality of“words.” In the embodiment shown in FIG. 3B, each word of the memorybank may be 512-bit wide and may contain 32 data units that each may be16 bits. Consecutive data units of a thread “i” may be referred to asSi(0),Si(1), . . . , and stored in a memory bank-i. For example, dataunits S0(0), S0(1), through S0(31) for thread zero (“0”) may be storedin the first word in the memory bank 302.1, and data units S0(32),S0(33), through S0(63) for thread 0 may be stored in the second word inthe memory bank 302.1, and so on. Similarly, data units S1(0), S1(1),through S1(31) for thread one (“1”) may be stored in the first word inthe memory bank 302.2, and data units S1(32), S1(33), through S1(63) forthread 1 may be stored in the second word in the memory bank 302.2, andso on. And data units S31(0), S31(1), through S31(31) for thread 31 maybe stored in the first word in the memory bank 302.N, and data unitsS31(32), S31(33), through S31(63) for thread 31 may be stored in thesecond word in the memory bank N−1, and so on.

In one embodiment of this first memory mapping, data units for differentthreads may be intended to be stored in different memory banks and wrapback to the first bank for thread N. For example, for N equal to 32,data units for the 32^(nd) thread may be stored to memory bank 0 (e.g.,data units S32(0) through S32(31) in memory bank 0), data units for the33^(rd) thread may be stored to memory bank 1 (e.g., data units S33(0)through S33(31) in memory bank 1), data units for the 63^(rd) thread maybe stored to memory bank N−1 (e.g., data units S63(0) through S63(31) inmemory bank 0), and so on.

For the same memory structure of FIG. 3B, data could be mapped in adifferent way. FIG. 3C schematically shows a second memory mapping for aprivate memory access mode in accordance with an embodiment of thepresent disclosure. The memory unit 300 shown in FIG. 3C may comprisethe same plurality of memory banks as in FIG. 3A, and each word in thememory banks 302.1-302.N of FIG. 3C may also be 512 bits wide and eachdata unit be 16 bits wide. Consecutive data units of a thread i maystill be stored in a memory bank-i but in different words. For example,data units S0(0), S0(1) and so on for thread 0 may be stored indifferent words in a column direction in memory bank 302.1; data unitsS1(0), S1(1) and so on for thread 1 may be stored in different words ina column direction in memory bank 302.2; data units S31(0), S31(1) andso on for thread 31 may be stored in different words in a columndirection in memory bank 302.N; and so on.

In one embodiment of this second memory mapping, data units fordifferent threads may be intended to be stored in different memory banksand wrap back to the first bank for thread N and integer multiple of N(e.g., 2N, 3N, etc.). Moreover, data units of a group of differentthreads with same index may be mapped to the same word of a memory bank.For example, for N equal to 32, data units for the 32^(nd) thread may bestored to memory bank 302.1 in different words (e.g., data units S32(0)through S32(99) in memory bank 302.1 in a second column, with data unitsS0(m) and S32(m) in the same word, m being the index of the data unit ina thread), data units for the 33^(rd) thread may be stored to memorybank 302.2 in different words (e.g., data units S33(0) through S33(99)in memory bank 302.2 in a second column, with data units S1(m) andS33(m) in the same word, m being the index of the data unit in athread), data units for the 63^(rd) thread may be stored to memory bank302.N (e.g., data units S63(0) through S63(99) in memory bank 0, withdata units S31(m) and S63(m) in the same word, m being the index of thedata unit in a thread), and so on. Because each word has 32 data units,the last data unit in the first row of memory bank 302.1 may be thefirst data unit S992(0) of the thread 992, the last data unit in thefirst row of memory bank 302.2 may be the first data unit S993(0) of thethread 993, and so on until the last data unit in the first row ofmemory bank 302.N may be the first data unit S1023(0) of the thread1023. It should be noted that a thread may have more than 99 data unitsand Si(99) (e.g., S0(99), etc.) may not be the last data units for athread and dotted lines may represent that more data units may exist andstored in a memory bank.

The data units for thread 1024 and higher number of threads may bewrapped from the first column of memory bank 0 and so on. For example,with m being the index, data units for threads 1024, 1056 and so onuntil 2016 (e.g., S1024(m), 51056(m) and so on until S2016(m)) may be inone word of the memory bank 0; data units for threads 1025, 1057 and soon until 1057 (e.g., S1025(m), S1057(m) and so on until S2017(m)) may bein one word of the memory bank 1; and data units for threads 1055, 1087and so on until 2047 (e.g., S1055(m), S1087(m) and so on until S2047(m))may be in one word of the memory bank N−1.

FIG. 3D schematically shows one example of memory mapping for a sharedmemory access in accordance with an embodiment of the presentdisclosure. The memory unit 300 shown in FIG. 3D may comprise the sameplurality of memory banks as in FIG. 3A, each word in the memory banks302.1-302.N of FIG. 3D may also be 512 bits wide and each data unit be16 bits wide. In this example memory mapping for shared memory access,consecutive data units A(0),A(1), . . . , may be stored in differentmemory banks in interleaved fashion. For example, for N equal to 32,A(0), A(1), A(2), and so on may be spread out in N memory banks, withA(0) in memory bank 0, A(1) in memory bank 1, and so on until A(31) inmemory bank N−1; and wrap back with A(N) in memory bank 0 in the sameword as A(0), A(N+1) in memory bank 1 in the same word as A(1), and soon, until A(N+31) in memory bank N−1 in the same word as A(31); and wrapback and so on, until A(992) (e.g. A(31N)) in memory bank 0 in the sameword as A(0), A(993) (e.g., A(31N+1)) in memory bank 1 in the same wordas A(1), and so on until A(1023) (e.g., A(31N+31)) memory bank N−1 inthe same word as A(31). After one word of a memory bank may be filled,more consecutive data units may be spread in another word in the memorybanks. For example, A(1024), A(1056), through A(2016) may be in anotherword in memory bank 0; A(1025), A(1057), through A(2017) may be inanother word in memory bank 1; and so on, until A(1055), A(1087),through A(2047) may be in another word in memory bank N−1.

Regardless of private or shared memory access modes, each of the caches304.1-304.N of a memory unit 300 may comprise multiple cache lines thateach may temporarily store a memory word from a corresponding memorybank. For example, cache 304.1 may comprise multiple cache lines thateach may be configured to temporarily store one word retrieved from thememory bank 302.1 (e.g., memory bank 0), cache 304.2 may comprisemultiple cache lines each configured to temporarily store one wordretrieved from the memory bank 302.2 (e.g., memory bank 1), cache 304.Nmay comprise multiple cache lines each configured to temporarily storeone word retrieved from the memory bank 302.N (e.g., memory bank N−1),and so on. A cache miss may be generated when one or more data pieces(e.g., one or more data units) requested are not in the cache. In oneembodiment, one memory word of a memory bank of the memory unit 300(e.g., in either FIG. 3B, FIG. 3C or FIG. 3D) may be fetched as onecache line into a cache when there is a cache miss. Generally, thelarger the cache size, more cache lines may be used to store memorywords and the lower cache miss rate may be expected. In someembodiments, storage units in a cache may be implemented as registers.

Data storage in the memory unit 212 may be accessed by the MPs220.1-220.N via the caches 304.1-304.N. The memory ports (MP) at eachcolumn may be configured with same components to carry out the memoryoperations, for example, calculating addresses and issuing read and/orstore operations. In some embodiments, one cache 304 may be accessed bymultiple MPs at the same time. Each of the MPs may be configured toprovide the two accessing modes: the private memory access mode and theshared memory access mode. Due to the nature of SIMT, memory read orwrite instructions mapped to a MP for different threads belong to thesame type, either shared or private. Moreover, a MP may be configuredfor private or shared memory access mode for a duration of aconfiguration.

FIG. 4A schematically shows a first memory access configuration for amemory port (MP) 400 in accordance with an embodiment of the presentdisclosure. The MP 400 may be an embodiment of a memory port 220. Thefirst memory access configuration shown in FIG. 4A may be one exampleconfiguration for the private memory access mode. During operation, theMP 400 may receive a memory port (MP) configuration for a physical datapath from the sequencer and the MP configuration may specify that forthis physical data path the MP 400 may be configured for the privatememory access mode. The MP 400 may comprise an address calculation unit402 and a counter 404. The address calculation unit 402 may take a baseaddress as a first input, an offset as a second input, and a third inputfrom the counter 404. The base address may be a common address for allthreads. The offset may be coupled to an address input port of the MP400, which may be configured to accept a vector address. The vectoraddress may comprise individual addresses for each concurrent thread andthese addresses may be referred to as address offsets. In oneembodiment, the base address in the first memory access configurationmay contain the starting address of the thread-0 data unit 0, e.g.,S0(0). The number of concurrent threads may be limited by the number ofALUs in a PE, and the width of the vector address and vector data bus.For example, if the number of ALUs in a ALU vector of a PE is N, and thevector address may comprise N addresses and vector data bus may compriseN data buses, there may be N concurrent threads. The offset input may bethe vector address for the N-threads. Each address offset (e.g., eachaddress of the vector address) may be independentlyprogrammed/calculated.

The third input from the counter 404 may provide thread numbers (e.g.,indexes) for the address calculation unit 402 and therefore, the counter404 may be referred to as a thread counter. In one embodiment, theaddress vector, read data vector and write data vector may be simplysplit into each memory bank with a one-to-one mapping so that the dataof different threads may be mapped into different memory banks. Forexample, the i-th address in the vector address may be for thread i(lower case letter “i” to denote a thread number, which may start fromzero for the first thread), and the counter 404 may provide a threadnumber vector to the address calculation unit 402 so the addresscalculation unit 402 may generate N addresses as A_0, A_1 . . . , A_N−1in this example corresponding to the vector size of ALU. Each address inthe vector address may be mapped to an address A_i and a correspondingaddress output for a corresponding memory bank (e.g., A_0 coupled to theaddress port 410.1 for the memory bank 0 cache 304.1, A_N−1 coupled tothe address port 410.N for memory bank N−1 cache 304.N, etc.). The i-thdata lines in the vector write data port WData 406 may be mapped to WD_i(e.g., WD_0 coupled to the write data port 412.1 for memory bank 0 cache304.1, WD_N−1 coupled to the write data port 412.N for memory bank N−1cache 304.N, etc.). The i-th data lines in the vector read data portRData 408 may be mapped to RD_i (e.g., RD_0 coupled to the read dataport 414.1 for memory bank 0 cache 304.1, RD_N−1 coupled to the readdata port 414.N for memory bank N−1 cache 304.N, etc.). No bus switchmay be needed for this configuration and there may be no memorycontention at this level.

It should be noted that the number of memory banks does not need to beidentical to the vector size. For example, a vector (e.g., vector ALU,vector address, vector data ports) may have a vector size=V, a PE arraymay have a number of columns=N, and a memory unit may have a number ofmemory banks=M, and V, N and M may be all different. For convenience,the capital letter N may be used herein to denote the vector size, thenumber of columns of PEs, and the number of memory banks, but the numberrepresented by N may be equal or different in different components.

For thread numbers larger than the number N, the address calculationunit 402 and the counter 404 may generate a memory mapping that wrapsaround to N memory banks. For example, thread 32 may be mapped to memorybank 0 cache 304.1 (e.g., S32(0) to memory bank 302.1 in FIGS. 3B and3C), thread 63 may be mapped to memory bank N−1 cache 304.N (e.g.,S63(0) to memory bank 302.N in FIGS. 3B and 3C).

FIG. 4B schematically shows a second memory access configuration for theMP 400 in accordance with an embodiment of the present disclosure. Thesecond memory access configuration shown in FIG. 4B may be one exampleconfiguration for the shared memory access mode. During operation, theMP 400 may receive a memory port (MP) configuration for a physical datapath from the sequencer and the MP configuration may specify that forthis physical data path the MP 400 may be configured for the sharedmemory access mode. The address calculation unit 402 may take the baseaddress as a first input and the offset as a second input, same as thefirst memory access configuration in FIG. 4A. But the counter 404 is notused in the shared memory access mode and the input from the counter 404may be ignored. The base address of the shared memory is common to allthe threads but the offsets could be different from each thread. Theaddress calculation unit 402 may generate N addresses as A_0, A_1 . . ., A_N 1 in this example corresponding to the vector size of ALU. Incontrast to the first memory access configuration in FIG. 4A, in whicheach address A_i may be mapped to one memory bank, in the second memoryaccess configuration, the N addresses from the address calculation unit402 may be delivered to a plurality of address selection units (e.g.,“Select 2” units 416.1 through 416.N). Each address selection unit 416.1through 416.N may also take a mask as an input, for example, “Bank 0”, .. . , and “Bank N−1”, and addresses targeted to a particular memory bankmay be arbitrated by a respective selection unit such that a fewaddresses may be selected. The upper number of addresses that may beselected may depend on design considerations, for example, 2, 3 or more.And because the vector address may have a fixed number of addresses, ifmore than one address is directed to one memory bank, there may be oneor more memory banks that have no addresses directed to. The selectedaddresses may be mapped to a memory port for a memory bank cache (e.g.,address port 426.1 for memory bank 0 cache 304.1, address port 426.N formemory bank N−1 cache 304.N, etc.). In one embodiment, for example, upto two addresses may be selected from N addresses and each address port426.1 through 426.N may be configured to deliver up to two memoryaddresses for a corresponding memory bank.

Because more than one address may be selected for one memory bank, writedata selection units (e.g., “Select 2” units 418.1 through 418.N) andread data selection units (e.g., “Select” units 420.1 through 420.N) maybe provided to map multiple data ports from the vector data ports WData406 and RData 408 to one memory bank. Each of the write data selectionunit 418.1 through 418.N may take an input from a corresponding dataselection unit 416.1 through 416.N, and map multiple write data linesfrom the write data lines WD_0 through WD_N−1 to a corresponding writedata port for a selected memory bank (e.g., write data port 422.1 formemory bank 0 cache 304.1, write data port 422.N for memory bank N−1cache 304.N). Each of the read data selection unit 420.1 through 420.Nmay take an input from a corresponding data selection unit 416.1 through416.N passed over by a corresponding selection unit 418.1 through 418.N,and map multiple read data lines from the read data lines RD_0 throughRD_N−1 to a corresponding read data port for a selected memory bank(e.g., read data port 424.1 for memory bank 0 cache 304.1, read dataport 424.N for memory bank N−1 cache 304.N). In an embodiment in whichup to two addresses may be selected from N addresses, the width of theaddress ports 426.1 through 426.N, the write data ports 422.1 through422.N and the read data ports 424.1 through 424.N may be doubled of thatof the address ports 410.1 through 410.N, the write data ports 412.1through 412.N and the read data ports 414.N.

Embodiments of a processor may comprise a large amount of ALUs andsupport massive amount of parallel threads. The memory access could bevery busy. It may be extremely expensive to use multiport memory to meetthe requirement. The complexity may also become very high if largeamount of memory banks is used. The example private memory access mayreduce the complexity of memory structure and support many typicalmemory access patterns for parallel processing. Some typical privatememory access patterns are listed below.

TABLE 1 PRIVATE MEMORY ACCESS PATTERNS Memory Access Pattern for PrivateMemory Access 1) One column mode: One MP may access the memory unit forall threads (all executing same instruction) in one PE, arbitrary andindependent offset may be used for each ALU thread. I.e., {a0 = C0, a1 =C1, . . . , a31 = C31}, where C0 may be the address offset for thread-0,C1 may be the address offset for thread-1. There may be not memorycontention in this case due to explicit memory partitioning fordifferent threads accessing data located in different memory banks. 2)Linear mode: Multiple MPs may access the memory unit for concurrentthreads in different PEs, address offsets for each MP may be linear. Forexample, for concurrent threads in a first PE, the address offsets maybe {a0 = C, a1 = C, . . . , a31 = C} for a first MP, for concurrentthreads in a second PE, the address offsets may be {b0 = C + 1, b1 = C +1, . . . , b31 = C + 1} for a second MP, and so on for other concurrentthreads in other PEs. 3) Reverse linear mode: Multiple MPs may accessthe memory unit for concurrent threads in different PEs, address offsetsfor each MP may be reverse linear. For example, for concurrent threadsin a first PE, the address offsets may be {a0 = C, a1 = C, . . . , a31 =C} for a first MP, for concurrent threads in a second PE, the addressoffsets may be {b0 = C − 1, b1 = C − 1, . . . , b31 = C − 1}for a secondMP, and so on for other concurrent threads in other PEs 4) Overlap mode:Multiple MPs may access the memory unit for concurrent threads indifferent PEs, address offsets for each MP may be overlapped. Forexample, for concurrent threads in a first PE, the address offsets maybe {a0 = C, a1 = C + 1, . . . , a31 = C + 31}for a first MP, forconcurrent threads in a second PE, the address offsets may be {b0 = C +1, b1 = C + 2, . . . , b31 = C + 32} for a second MP, and so on forother concurrent threads in other PEs 5) Non-unity stride mode: MultipleMPs may access the memory unit for concurrent threads in different PEs,address offsets from each MP may be spaced by a stride D. For example,for concurrent threads in a first PE, the address offsets may be {a0 =C, a1 = C, . . . , a31 = C} for a first MP, for concurrent threads in asecond PE, the address offsets may be {b0 = C + D, b1 = C + D, . . . ,b31 = C + D} for a second MP, for concurrent threads in a third PE, theaddress offsets may be {c0 = C + 2D, c1 = C + 3D, . . . , c31 = C + 3D}for a third MP, and so on for other concurrent threads in other PEs 6)Random mode: Multiple MPs may access the memory unit for concurrentthreads in different PEs, address offsets from each MP may be randomnumbers. For example, for concurrent threads in a first PE, the addressoffsets may be {a0 = n_0, 0, a1 = n_0, 1, . . . , a31 = n_0, 31} for afirst MP, for concurrent threads in a second PE, the address offsets maybe {b0 = n_1, 0, b1 = n _2, 0, . . . , b31 = n_31, 0} for a second MP,and so on for other concurrent threads in other PEs, where n_(i, j) maybe different random numbers.

In some embodiments, the private memory access may allow random dataaccess from all threads at the same time but to different memory areafor each thread. This enables programmers to write software inconventional style, without complicated data vectorization and detailedknowledge of underlying processor hardware architecture. This may enablesame-instruction-multiple-thread (SIMT) programming to be applicable toan embodiment of a PE array. That is, one instruction may beconcurrently executed by multiple threads in one PE.

Due to the non-overlapping nature, the total throughput may be the sumof throughputs of all threads. Embodiments of the private memory accessmode may support large throughput from simultaneous access from eachthread. The first and second memory data mapping may allow minimummemory contention in typical private data access patterns. Embodimentsof private memory access may also reduce the complexity of memorysystem. The number of memory banks may be significantly reduced. Theparallel cache structure may also reduce the total cache size since eachcontent in the cache may be unique. Moreover, embodiments of privatememory access may significantly reduce access to the memory banks byallowing simultaneous cache access from multiple memory ports.

In one embodiment, for a PE array size with 32×32 ALUs, only 32 memorybanks may be required using the private memory access configuration(e.g., as shown in FIG. 4A). This may greatly reduce the number ofmemory banks from 1024 memory banks for a conventional design to supportdifferent addresses used by each thread.

Different memory access patterns may use different mapping methods, bothmappings in FIG. 3B and FIG. 3C may be supported by an embodiment of thememory access configuration shown in FIG. 4A using an address generationinstruction.

The first memory mapping in FIG. 3B may be good to handle case-1,case-2, case-3 and case-4 in Table 1. In case-6 in Table 1, if the rangeis within the cache size, it also can be handled well. The second memorymapping in FIG. 3C may be good to handle case-1, case-2, case-3, andcase-5 in Table 1.

In some embodiments, register spilling may occur. Register spilling mayrefer to scenarios that when a compiler is generating machine code,there are more live variables than the number of registers the machinemay have and thus some variables may be transferred or spilled tomemory. Memory for register spilling may be private to each thread,these spilled variables may need to be stored in private memory. Due tothe fact that all address offset for register spilling may be identicalto each thread, they are similar to Non-unity stride mode in case-5 ofTable 1 and the spilled variables may be stored using the second memorymapping as shown in FIG. 3C, and there may be no memory contention.

The example shared memory access mode may also reduce the complexity ofmemory structure and support many typical memory access patterns forparallel processing. Some typical shared memory access patterns arelisted below.

TABLE 2 SHARED MEMORY ACCESS PATTERNS Memory Access Pattern for SharedMemory Access 1) One MP may access the memory unit for all threads inone PE, same addresses for all these concurrent threads. For example,the address offsets may be {a0 = C, a1 = C, . . . , a31 = C} for theconcurrent threads 2) Multiple MPs may access the memory unit forconcurrent threads in different PEs, address offsets may be the samewithin one MP but different for different MPs. For example, forconcurrent threads in a first PE, the address offsets may be {a0 = C1,a1 = C1, . . . , a31 = C1} for a first MP, for concurrent threads in asecond PE, the address offsets may be {b0 = C2, b1 = C2, . . . , b31 =C2} for a second MP, and so on for other concurrent threads in other PEs3) One MP may access the memory unit for all threads in one PE, theaddresses may be sequential, for example, the address offsets may be {a0= C, a1 = C + 1, . . . , a31 = C + 31} for the concurrent threads 4)Multiple MPs may access the memory unit for concurrent threads indifferent PEs, for example, for concurrent threads in a first PE, theaddress offsets may be {a0 = C1, a1 = C1 + 1, . . . , a31 = C1+ 31} fora first MP, for concurrent threads in a second PE, the address offsetsmay be b{b0 = C2, b1 = C2 + 1, . . . , b31 = C2 + 31} for a second MP,and so on for other concurrent threads in other PEs 5) One MP may accessthe memory unit for all threads in one PE, address may be sequentialwith discontinuity, for example, the address offsets may be {a0 = C1, a1= C1 + 1, a2 = C1 + K1, a3 = C2 + K1 + 1 . . . , } for the concurrentthreads 6) Multiple MPs may access the memory unit for differentconcurrent threads in different PEs, address may be sequential withdiscontinuity, for example, for concurrent threads in a first PE, theaddress offsets may be {a0 = C1, a1 = C1 + 1, a2 = C1 + K1, a3 = C2 +K1 + 1 . . . , } for a first MP, for concurrent threads in a second PE,the address offsets may be {b0 = C2, b1 = C2 + 1, b2 = C2 + K2, b3 =C2 + K2 + 1 . . . , } for a second MP, etc. 7) One MP may access thememory unit for all threads in one PE, address may be linear withnon-unity stride, for example, the address offsets may be {a0 = C1, a1 =C1 + K, a2 = C1 + 2K, a3 = C2 + 3K . . . , } for the concurrent threads8) One MP may access the memory unit for all threads in one PE,addresses may be random but within a small range, for example, theaddress offsets may be {a0 = n0, a1 = n1, a2 = n2, a3 = n3 . . . , },where n0, n1, n2, . . . , ∈ (C, C + R), which may mean n0, n1, n2, . . ., are in the range of C to C + R, the range C to C + R may be what canbe stored in the overall memory cache (e.g., R may be the capacity ofall memory caches combined, and data stored in memory addresses C to C +R may be loaded and distributed across all memory caches at the sametime) 9) Multiple MPs may access the memory unit for concurrent threadsin different PEs, addresses may be random but with a small range, forexample, for concurrent threads in a first PE, the address offsets maybe {a0 = n00, a1 = n01, a2 = n02, . . . , } for a first MP, forconcurrent threads in a second PE, the address offsets may be {b0 = n10,b1 = n11, b2 = n12, . . . , } for a second MP, etc., where n00, . . . ,n3131 ∈ (C, C + R)

In some embodiments, the shared memory access may allow random dataaccesses from each parallel thread at the same time. All threads mayaccess anywhere in a common area in the memory unit. In one embodiment,the common area may be a shared memory space that includes all memorybanks. In another embodiment, the common area may be a shared memoryspace across a plurality of memory banks. This may enable programmers towrite software in conventional style, without complicated datavectorization and detailed knowledge of underlying processor hardwarearchitecture. This may also enable SIMT programming to be applicable toan embodiment of a PE array.

Embodiments of shared memory access may reduce the complexity of memorysystem. The number of memory banks may be significantly reduced. Theparallel cache structure may also reduce the total cache size since eachcontent in the cache may be unique. Moreover, embodiments of sharedmemory access may significantly reduce access to the memory banks byallowing simultaneous cache access from multiple memory ports.

In one embodiment, for a PE array size with 32×32 ALUs, only 32 memorybanks may be needed using the shared memory access configuration (e.g.,as shown in FIG. 4B). This may greatly reduce the number of memory banksfrom 1024 memory banks for a conventional design to support differentaddresses used by each thread.

It should be noted that as shown in FIG. 2, each MP may provide two readdata ports (two arrows from the memory unit 212 to a MP and two arrowsto a corresponding SB, e.g., for MP 220.1, two arrows from the memoryunit 212 at the top and two arrows to SB 222.1 at the bottom) and twowrite data ports (two arrows to the memory unit 212 and two arrows froma corresponding SB, e.g., for MP 220.1, two arrows from the memory unit212 at the top and two arrows to SB 222.1 at the bottom), in addition totwo inputs and outputs from and to a next MP or the gasket memory 216(e.g., two horizontal arrows from left to right between the MPs, andfrom MP 220.N to the gasket memory 216 and from the gasket memory 216 tothe MP 220.1). That is, in some embodiments, a MP 400 may have aduplicate set of components shown in FIGS. 4A and 4B to provide twoconcurrent memory access for two concurrent data access operations, forexample, two reads, two writes, or one read and one write. In someembodiments, the two concurrent data access operations may be vectordata operations. That is, the two read operations may be two vector datareads (e.g., reading 2N pieces of data, with N being the number ofconcurrent threads), the two write operations may be two vector datawrites (e.g., writing 2N pieces of data, with N being the number ofconcurrent threads), and the one read and one write operations may beone vector data read and one vector data write (e.g., reading N piecesof data and writing N pieces of data with N being the number ofconcurrent threads). In one embodiment, for example, to performconcurrent one read and one write operations, a MP may read N pieces ofdata for a PE in the same column and at the same time write N pieces ofdata generated from a PE in a preceding column.

FIG. 5 schematically shows a switch box (SB) 500 in accordance with anembodiment of the present disclosure. The SB 500 may be an embodiment ofa SB 222 and may comprise a plurality of data inputs and data outputs,and inter-connections that couple the data outputs to data inputs fordata switching. The data inputs of SB 500 may comprise data inputs502.1, 502.2, 512.1, 512.2, 514.1 and 514.2. The data outputs of SB 500may comprise data outputs 504.1, 504.2, 506.1, 506.2, 508.1, 508.2,510.1 and 510.2. The SB 500 may further comprise a configuration buffer518 and a corresponding configuration input 516. The configurationbuffer 518 may be implemented as a First-In-First-Out buffer andreferred to as C-FIFO 518. The configuration input 516 may be coupledexternally to the configuration bus that's coupled to the sequencer 206for the SB 500 to receive configurations from the sequencer 206. Theconfigurations for the SB 500 may be referred to as SB configurations.Moreover, the SB 500 may further comprise a plurality of counters520.1-520.8 each corresponding to a data output, for example, counter520.1 for data output 504.1, counter 520.2 for data output 504.2,counter 520.3 for data output 506.1, counter 520.4 for data output506.2, counter 520.5 for data output 508.1, counter 520.6 for dataoutput 508.2, counter 520.7 for data output 510.1, and counter 520.8 fordata output 510.2.

Each data input of the SB 500 may be coupled to some data outputs. Forexample, the data input 502.1 may be coupled to the data outputs 506.1,506.2, 508.2, 510.1 and 510.2; the data input 502.2 may be coupled tothe data outputs 506.1, 506.2, 508.1, 510.1 and 510.2; the data input512.1 may be coupled to the data outputs 504.1, 504.2, 506.1, 506.2, and508.1; the data input 512.2 may be coupled to the data outputs 504.1,504.2, 506.1, 506.2, and 508.2; the data input 514.1 may be coupled tothe data outputs 504.1, 506.1, 506.2, 508.1, and 510.2; and the datainput 514.1 may be coupled to the data outputs 504.2, 506.1, 506.2,508.2, and 510.1.

Externally, depending on the location of the SB 500 in the PE array 214,the data inputs 502.1 and 502.2, and data outputs 504.1 and 504.2 may becoupled to a MP 220, or another SB 222 (e.g., in a multi-row PE array).The data inputs 514.1 and 514.2 may be coupled to a PE 218 or the gasketmemory 216. The data inputs 512.1 and 512.2, and data outputs 510.1 and510.2 may be coupled to another SB 222 (e.g., in a multi-row PE array)or an ICSB 224. The data outputs 506.1, 506.2, 508.1 and 508.2 may becoupled to a PE 218. Data signals output from the data outputs 506.1,506.2, 508.1 and 508.2 may be denoted as A, B, C, D, and data signalsinput from the data inputs 514.1 and 514.2 may be denoted as X, Y. Thesedata signals A, B, C, D, and X, Y may be the input data signals to a PE218 and output data signals from a PE 218 as described herein.

Each of the counters 520.1-520.8 at the data outputs may beindependently responsible for counting data passed. When one or moreconfigurations may be loaded into the C-FIFO 518, each configuration mayspecify a number of counts. During execution of one configuration, allcounters may independently count how many times data has passed through.When all the counters reach the number of counts specified in theconfiguration, a next configuration may be applied. A similar approachmay be applied inside an ICSB 224, a PE 218, the gasket memory 216 and amemory port 220. Because these counters may facilitate configuration andreconfiguration of each component that may have a such counter, thesecounters may be referred to as reconfiguration counters and a componentthat has such a counter may be referred to as a reconfigurable unit. Anembodiment of a processor 200 may provide massive parallel dataprocessing using the various reconfigurable units and may be referred toas a reconfigurable parallel processor (RPP).

FIG. 6 schematically shows a processing element (PE) 600 in accordancewith an embodiment of the present disclosure. The PE 600 may be anembodiment of a PE 218. The PE 600 may comprise an arithmetic logic unit(ALU) 602, a plurality of data buffers (e.g., D-FIFO 604.1, 604.2, 604.3and 604.4), a counter 606, a plurality of data outputs (e.g., 608.1 and608.2), a plurality of data inputs (e.g., 610.1, 610.2, 610.3 and610.4), a configuration input 612 and a configuration buffer (e.g.,C-FIFO) 614). In one embodiment, the ALU 602 may be one ALU (e.g., oneALU that is configured to process one piece of data at a time and may bereferred as a scalar ALU). In some other embodiments, the ALU 602 may bea vector of ALUs (or referred to as a vector ALU), for example, N ALUs(with N may be referred to as the vector size of ALU), and SameInstruction Multiple Data (SIMD) operation may be applied to all ALUs ofthe vector. Please note that a scalar ALU may be a special case of avector ALU with a vector size being one.

Data signals received from the data inputs 610.1, 610.2, 610.3 and 610.4may be denoted as A, B, C, D, and data signals output from the dataoutputs 608.1 and 608.2 may be denoted as X, Y. In an embodiment inwhich the ALU 602 may be one ALU, each data input 610.1, 610.2, 610.3 or610.4 and each data output 608.1 or 608.2 may have a width of M bitsthat may match the width of the ALU. For example, for an 8-bit ALU, eachinput and output may be 8-bit; for a 16-bit ALU, each input and outputmay be 16-bit; for a 32-bit ALU, each input and output may be 32-bit;and so on. And each input data signal A, B, C, D and each output signalX, Y may be M bits. In an embodiment in which the ALU 602 may be avector of ALUs, each data input 610.1, 610.2, 610.3 or 610.4 may be avector of N M-bit inputs, and each data output 608.1 or 608.2 may be avector of N M-bit outputs. And each input data signal A, B, C, D andeach output data signal X, Y may be N×M bits.

The data buffers 604.1-604.4 may be coupled to the inputs 610.1, 610.2,610.3 and 610.4 to temporarily store data pieces. In some embodiments,however, the data buffers may be located as the output. The D-FIFOs604.1-604.4 may be used to decouple the timing of PEs to allow PEs towork independently. In one embodiment, the buffers may be implemented asFIFOs (e.g., a D-FIFO for a data buffer, a C-FIFO for a configurationbuffer).

The configuration buffer C-FIFO 614 may receive configurations from theconfiguration input 612, which may be coupled externally to thesequencer 206 via the configuration bus, and store the receivedconfigurations before any execution of a data path starts. Theconfigurations for the PE 600 may be referred to as PE configurations.The PE 600 may be statically programmed with instructions for oneconfiguration, e.g., the instructions may be programmed to the PE 600 asa stage of pipeline. No instructions may be changed during oneconfiguration. Once configured, the operation of the ALU 602 (e.g., oneALU or vector of ALUs depending on a particular embodiment) may betriggered if D-FIFOs 610.1, 610.2, 610.3 and 610.4 have data and outputports 608.1 and 608.2 are not busy. One of the configuration parametermay be a number for a specified number of executions for an instruction.The counter 606 may be programmed with the specified number and used tocount the number of times data has been processed by executing aninstruction. When the number of executions has reached the specifiednumber, a new configuration may be applied. Therefore, reconfigurationcapability may be provided in each PE. In one embodiment, the specifiednumber of execution for an instruction may be referred to as NUM_EXECand this NUM_EXEC may be used across a data path for one configuration.

In one embodiment with a multi-row PE array 214, the PEs within eachcolumn may be functionally different from each other but the PEs alongeach row follow a repetitive pattern (e.g., functionally duplicative).For example, ALUs in a first row of PEs may implement a first set ofinstructions and ALUs in a second row of PEs may implement a second setof instructions that may be different from the first set. That is, ALU602 in different embodiments of the PE 600 may comprise differentstructures or different functional components. In some embodiments, oneor more rows of PEs of a processor may comprise ALUs that may berelatively simple and use less space and another row of PEs of the sameprocessor may comprise ALUs that may be relatively more complex and usemore space. The relatively simple ALUs may implement a set ofinstructions that may be different from a set of instructionsimplemented by the relatively more complex ALUs. For example, oneembodiment of PE 600 may have an ALU 602 (e.g., one ALU or a vector ofALUs) that implements a set of instructions that require a relativelysimple structure, such as, but not limited to, ADDITION (e.g., A+B),SUBSTRACTION (e.g., A-B), etc.; while another embodiment of PE 600 mayhave an ALU 602 that implements instructions that require a relativelymore complex structure, such as, but not limited to, MULTIPLICATION(e.g., A times B (A*B)), MAD (for multiply-accumulate (MAC) operation)(e.g., A*B+C).

FIG. 6B schematically shows another processing element (PE) 620 inaccordance with another embodiment of the present disclosure. Theprocessing element 620 may be alternative embodiment of a PE 218. Asshown in FIG. 6B, the PE 620 may have a structure similar to theprocessing element 600 and differ by having a plurality of indicators“S” 622.1 through 622.4 that each of the indicators may be associatedwith one respective D-FIFO. For example, the indicator 622.1 may beassociated with D-FIFO 604.1, the indicator 622.2 may be associated withD-FIFO 604.2, the indicator 622.3 may be associated with D-FIFO 604.3and the indicator 622.4 may be associated with D-FIFO 604.4. Theseindicators may be used for static read configuration (also referred toas static load configuration). For example, during one configuration, aninstruction may be set to be executed a specified number of times (e.g.,NUM_EXEC=3) by a PE according to the configuration. Executing theinstruction may use data from all four D-FIFOs 604.1 through 604.4. Theindicator 622.2 may be set while other indicators 622.1, 622.3 and 622.4may not be set. The data in the D-FIFO 604.2 may be re-used while theinstruction is repeated, but new data in the D-FIFOs 604.1 604.3 and604.4 may be used for each repetition of the instruction.

FIG. 7 schematically shows an inter-column switch box (ICSB) 700 inaccordance with an embodiment of the present disclosure. The ICSB 700may be an embodiment of an ICSB 224 and may comprise a plurality of datainputs and data outputs, and inter-connections that couple the dataoutputs to data inputs for data switching. The data inputs of ICSB 700may comprise data inputs 704.1, 704.2, and 710.1-710.4. The data outputsof ICSB 700 may comprise data outputs 706.1, 706.2, and 708.1-708.4. TheICSB 700 may further comprise a configuration buffer 702 and acorresponding configuration input 712. The configuration buffer 702 maybe implemented as a First-In-First-Out buffer and referred to as C-FIFO702. The configuration input 712 may be coupled externally to theconfiguration bus that's coupled to the sequencer 206. Moreover, theICSB 700 may further comprise a plurality of counters 714.1-714.6 eachcorresponding to a data output, for example, counter 714.1 for dataoutput 708.1, counter 714.2 for data output 708.2, counter 714.3 fordata output 708.3, counter 714.4 for data output 708.4, counter 714.5for data output 706.1, and counter 714.6 for data output 706.2.

Each data input of the ICSB 700 may be coupled to some selected dataoutputs. For example, the data input 704.1 may be coupled to the dataoutputs 708.1-708.4; the data input 704.2 may be coupled to the dataoutputs 708.1-708.4; the data input 710.1 may be coupled to the dataoutputs 706.1-706.2, and 708.1; the data input 710.2 may be coupled tothe data outputs 706.1-706.2, and 708.2; the data input 710.3 may becoupled to the data outputs 706.1-706.2, and 708.3; and the data input710.4 may be coupled to the data outputs 706.1-706.2, and 708.4.

Externally, the data inputs 704.1 and 704.2, and data outputs 706.1 and706.2 may be coupled to a SB 222. The data inputs 710.1-710.4 may becoupled to a neighboring ICSB 224 or the gasket memory 216. The dataoutputs 708.1-708.4 may be coupled to another neighboring ICSB 224 orthe gasket memory 216.

Each of the counters 714.1-714.6 at the data outputs may beindependently responsible for counting data passed. When one or moreconfigurations may be loaded into the C-FIFO 702, each configuration mayspecify a number of counts. The configurations for the ICSB 700 may bereferred to as ICSB configurations. During execution of oneconfiguration of the PE array 214, all counters may independently counthow many times data has passed through. When all the counters reach thenumber of counts specified in the configuration, a next configurationmay be applied. This implementation may be similar to what may beapplied inside a SB 222, a PE 218, the gasket memory 216 and a memoryport 220.

FIG. 8 schematically shows a gasket memory 800 in accordance with anembodiment of the present disclosure. The gasket memory 800 may be anembodiment of the gasket memory 216 shown in FIG. 2. The gasket memory800 may comprise a plurality of data buffers for temporary storage ofdata. The data buffers may be implemented as First-In-First-Out (FIFO)buffers and referred to as D-FIFOs (e.g., D-FIFO 802.1-802.F, F being aninteger equal to or larger than 8). In addition, the gasket memory 800may comprise a plurality of data inputs (e.g., 814.1-814.2, 816.1-816.2,and 818.1-818.4), a plurality of data outputs (e.g., 808.1-808.2,810.1-810.2, and 812.1-812.4), an input configuration buffer (e.g.,Input C-FIFO 804), an output configuration buffer (e.g., Output C-FIFO806), a configuration input 816, a plurality of input counters820.1-820.L (L being an integer equal to or larger than 4), and aplurality of output counters 822.1-822.4.

External connections from the MP 220.N, PE 218.N and ICSB 224.N may betaken as inputs at the data inputs 814.1-814.2, 816.1-816.2, and818.1-818.4, respectively. And external connections to the MP 220.1, SB222.1 and ICSB 224.1 may generate outputs at the data outputs808.1-808.2, 810.1-810.2, and 812.1-812.4 respectively. Theconfiguration input 816 may be externally coupled to the sequencer 206via the configuration bus for the gasket memory 800 to receiveconfigurations from the sequencer 206. The configurations for the gasketmemory 800 may be referred to as gasket memory configurations. Two typesof configurations may be received from the sequencer 206: inputconfigurations and output configurations. The input C-FIFO 804 may storeinput configurations for input ICSB ports 818.1-818.4 to be coupled tosome Data FIFOs selected from L D-FIFOs 802.5-802.F as inputs to theseselected D-FIFOs. The output C-FIFO 806 may store configurations forsome data FIFOs selected from L D-FIFOs 802.5-802.F to be coupled to theICSB ports 812.1-812.4.

The number of gasket D-FIFOs 802.5 through 802.F storing ICSB inputs maybe greater or equal to the number of input or output ICSB ports. In someembodiments, as described herein, there may be a data connection thatmay bypass at least a portion of a physical data path. For example, anexecution result generated by one PE 218 may not be needed by another PE218 in the same physical data path configuration but may be used in afuture configuration. These data signals for the execution result may berouted via a SB 222 and an ICSB 224 to the gasket memory 216 and storedin the D-FIFOs of the gasket memory 216 for the future configuration.Therefore, in some embodiments, the gasket memory 800 may have moreD-FIFOs than the number of input or output ports.

Each of the input counters 820.1-820.L at the data inputs and each ofthe output counters 822.1-822.4 at the data outputs may be independentlyresponsible for counting data passed. When one or more inputconfigurations and output configurations may be loaded into the inputC-FIFO 804 and output C-FIFO 806, each configuration may specify anumber of counts. During execution of one configuration, all countersmay independently count how many times data has passed through. When allthe counters reach the number of counts specified in the configuration,a next configuration may be applied.

FIG. 9A schematically shows an example dependency graph of an executionkernel in accordance with an embodiment of the present disclosure. Inone embodiment, a sequence of instructions that may be run in one loop(e.g., one routine or one sub-routine) may be referred to as anexecution kernel or simply a kernel. The instructions in the group mayhave some dependency. For example, the kernel may have instructionsdenoted as A through K in the dependency graph. Instruction A may be thefirst instruction in the dependency graph of the kernel. Instructions Band C may need an output from the instruction A. Instruction B's outputmay be needed by both instructions D and K. Instruction D also may needan output from instruction C. Instruction D's output may be needed byboth instructions E and F. Instruction F's output may be needed byinstructions G, H and I. In addition to the output from instruction F,instruction G may also need an output from instruction E. Instruction Jmay need outputs from instructions G, H, I, respectively. And finally,instruction K may need outputs from instructions B and I. The dependencygraph of kernel may be mapped to data paths to be executed by aprocessor in accordance with an embodiment of the present disclosure.

FIG. 9B schematically shows the dependency graph of execution kernel ofFIG. 9A mapped into a virtual data path (VDP) for a processor inaccordance with an embodiment of the present disclosure. In someembodiments, mapping of a dependency graph of a kernel to a processormay be constrained by multiple factors, for example, size of a PE array,connections between PEs, memory access capability etc. It should benoted that the processor 200 in FIG. 2 schematically illustrates oneembodiment of a processor with one row of PEs, but in the embodimentshown in FIG. 9B, the virtual data path for the kernel may be mapped toa processor with two (“2”) rows of PEs. In general, a processor inaccordance with an embodiment of the present disclosure may utilize a1-D or 2-D data path (including a processing element (PE) array andinterconnections) to process massive parallel data. Each data path maybe segmented into sections. In a 1-D data path, a section may include amemory port, a switch box, a PE and an ICSB in one column; and in a 2-Ddata path, a section may include a memory port, two or more switchboxes, two or more PEs and an ICSB in one column. The data path may bemade identical in each section. This allows the dependency graph of thekernel to be mapped to a virtual data path which may include as manyrepetition as needed (e.g., unlimited or even theoretically infinite) ofthe data path sections in one dimension. For example, instructions of akernel may be mapped to columns of PEs and extended in the row directionrepetitively (e.g., loop back from the beginning of the row if needed).

FIG. 9C schematically shows the virtual data path of FIG. 9B dividedinto physical data paths for a processor in accordance with anembodiment of the present disclosure. In one embodiment, in order tosimplify the compiler work, a physical data path (PDP) may be made tohave repetitive structure. For example, each column may be identical andeach PDP may comprise same amount of repetitive columns. As shown inFIG. 9C, the VDP of FIG. 9B may be divided into three PDPs (e.g., PDP1,PDP2 and PDP3) for a 2×2 PE array and thus, the three PDPs may have thesame structure. The 2×2 PE array may be the whole PE array of anembodiment of a RPP, or may be part of a N×N (e.g., N being 32 as anexample) PE array of another embodiment of a RPP. There may be manyconnections between PEs in one PDP (e.g., A to B and C, B to D, C to D,etc.), between two consecutive PDPs (e.g., D to E and F, G to J, H to J,F to I, etc.) and between non-consecutive PDPs (e.g., B to K). Withinone PDP, all PEs in the PE array may be applied a configuration for thecurrent PDP, and data from one instruction may flow to anotherinstruction directly according to the dependency. For example, a PE maybe configured to execute instruction A in PDP1 and data from this PE maydirectly flow from this PE to the PEs configured to execute instructionsB and C. Between PDPs, data from one PDP may flow into a gasket memoryfor temporal storage. The data may flow out from the gasket memory asinput to the next physical data path in a new configuration. Forexample, a PE may be configured to execute instruction B in PDP1 in oneconfiguration, and data from this PE may be stored to a gasket memoryand used as input for a PE configured to execute instruction K in PDP3in a future configuration.

FIG. 10 schematically shows a pipeline operation for a processor inaccordance with an embodiment of the present disclosure. The example ofpipeline operation shown in FIG. 10 may be the physical data paths ofFIG. 9C mapped to a 2×2 PE array of four PEs: PE0, PE1, PE2 and PE3. Theinstructions of A, B, C, . . . K of the virtual data path may be choppedinto PDPs: PDP1, PDP2 and PDP3 as shown in FIG. 9C. PE0 may beconfigured to execute instruction A in PDP1, instruction E in PDP2 andinstruction I in PDP3. PE1 may be configured to execute instruction B inPDP1, instruction F in PDP2 and instruction J in PDP3. PE2 may beconfigured to execute instruction C in PDP1, instruction G in PDP2 andinstruction K in PDP3. PE3 may be configured to execute instruction D inPDP1, instruction H in PDP2 and no instruction in PDP3.

During operation, all concurrent threads in one PE may execute the sameinstruction and each instruction may be executed multiple times in onePE as a pipeline stage. That is, each PE may be configured to execute aninstruction NUM_EXEC times as a pipeline stage. For example, in anembodiment that each PE may comprise a ALU vector with a vector size ofone, each instruction may be configured to execute 4 times by the ALUvector at each PE. The 4 times of execution may be represented by fourthreads processed with each thread in a different shade. For example, inPDP1, PE0 may be configured to execute instruction A four times, PE1 maybe configured to execute instruction B four times, PE2 may be configuredto execute instruction C four times and PE3 may be configured to executeinstruction D four times. In PDP2, PE0 may be configured to executeinstruction E four times, PE1 may be configured to execute instruction Ffour times, PE2 may be configured to execute instruction G four timesand PE3 may be configured to execute instruction H four times. In PDP3,PE0 may be configured to execute instruction I four times, PE1 may beconfigured to execute instruction J four times, PE2 may be configured toexecute instruction K four times and PE3 may have no instructionconfigured. In this embodiment, because there may be data dependencybetween different instructions, a thread executing an instruction thatdepends on another instruction may be executed later in time. Forexample, instruction B may depend on data from instruction A's executionresult and therefore, the first thread executing instruction B mayfollow the first thread executing instruction A in a later cycle, thesecond thread executing instruction B may follow the second threadexecuting instruction A in a later cycle, the third thread executinginstruction B may follow the third thread executing instruction A in alater cycle, and the fourth thread executing instruction B may followthe fourth thread executing instruction A in a later cycle. Due tostatic reconfiguration scheme and dependency of the instructions, therecould be some time lost during DPD reconfiguration, e.g., PE2 may haveone idle cycle during PDP1 to PDP2 transition. In an embodiment in whicheach PE has a vector ALU with the vector size N larger than 1, each PEmay execute N concurrent threads at a time, and each shaded thread inFIG. 10 may represent N concurrent threads.

In various embodiments, the gasket memory may provide a way to reducethe efficiency loss during reconfiguration. For example, even there maybe some idle slots during reconfiguration (e.g., reconfiguration of PE2between instruction C of PDP1 and instruction G in PDP2), if a largernumber of thread are used, the idle slots may be insignificant comparedto the total busy cycles.

FIG. 11A-11I schematically show a configuration process for a processorin accordance with an embodiment of the present disclosure. Theconfiguration process may illustrate a data path (DP) configuration andreconfiguration. The Sequencer unit (SEQ) 1102 may be an embodiment ofthe sequencer 206 and may be responsible to schedule theinstructions/configurations for each PE (e.g. PE_0 1104, PE_1 1106, andPE_2 1108), and switch box (SB) (e.g., SB_1 1110, SB_2 1112, ICSB_1 1114and ICSB_2 1116). Each of PEs 1104, 1106 and 1108 may be an embodimentof a PE 218. Each of the SBs 1110 and 1112 may be an embodiment of a SB222. Each of the ICSBs 1114 and 1116 may be an embodiment of an ICSB224. Each of the PEs and SBs (including the SBs and ICSBs) of FIG. 11Amay be coupled to the SEQ 1102 via a configuration bus (shown as thedotted lines from the SEQ 1102). In this example, a Virtual Data Path(VDP) of an execution kernel may be mapped into two Physical Data Paths(PDPs). It should be noted that this example kernel may be differentfrom the data path examples shown in FIGS. 9A-9C and FIG. 10. The PEsmay be denoted with an underline in front of a numeral, such as PE_0,PE_1 and PE_2 in FIGS. 11A-11I to distinguish the example PE0, PE1, PE2,and PE3 in FIG. 10.

In the example configuration process in FIGS. 11A-11I, the first PDP(e.g., which may be referred to as PDP_1 with an underline in front of anumeral to distinguish from the data path example of FIG. 9C) may usethe data routing PE_0→PE_1, PE_1→PE_2, and the second PDP (e.g., whichmay be referred to as PDP_2 to distinguish from the data path example ofFIG. 9C) may use a different routing PE_0→PE_1, (PE_0,PE_1)→PE_2. Thatis, in the first PDP, PE_1 may rely on PE_0's output for its operation,PE_2 may rely on PE_1's output for its operation; and in the second PDP,the PE_1 may rely on PE_0's output for its operation, PE_2 may rely onboth PE_1's output and PE_0's output for its operation.

FIG. 11A illustrates that, initially, the entire DP is not configuredyet. All PEs may be in a default status. In one embodiment, this defaultstatus may be implemented by a default operation instruction, calledSTALL. Although PEs may be programmed STALL, their respective input dataFIFOs may be empty and ready to receive data. The busy signals, however,may be set to be 0. Therefore, all the SBs may be stalled and no signalmay be routed. For example, all data connections between the componentsmay be shown in dotted lines to indicate that no data may pass through(e.g., the data connections between PE_0 1104 and SB_1110, between SB_11110 and ICSB_1 1114, between SB_1 1110 and PE_1 1106, between PE_1 1106and SB_2 1112, between SB_2 1112 and ICSB_2 1116, and between SB_2 1112and PE_2 1108).

FIG. 11B illustrates that the SEQ 1102 may start to load the firstinstruction INS1 into PE_0 1104, but SB_1 1110 and ICSB_1 1114 may stillbe stalled. For example, the configuration connection from SEQ 1102 toPE_0 1104 may be open and illustrated as dashed lines, in contrast, theother configuration connections may be closed and shown in dotted lines.Because INST1 may be the first instruction in this execution kernel andnot dependent on any other conditions, PE_0 1104 may be ready togenerate outputs but blocked by the output busy signal from SB_1 1110.At this stage, no data may pass through any data connections between thecomponents and the data connections may be shown in dotted lines.

As shown in FIG. 11C, at this stage, the SEQ 1102 may have programmedboth SB_1 1110 and ICSB_1 1114, and the data route from PE_0→PE_1 may beopened. For example, the configuration connection from SEQ 1102 to SB_11110 and ICSB_1 1114 may be open and illustrated as dashed lines, incontrast, the other configuration connections may be closed and shown indotted lines. PE_0 1104 may generate outputs and these outputs may berouted via SB_1 1110 (e.g., a dash dotted line inside the SB_1 1110 toshow internal connection open) to PE_1 1106. PE_1 1106 may receive theoutputs from PE_0 1104 and store these data signals in its data buffer(e.g., D-FIFOs) even when PE_1 1106 has not been configured yet. Thedata connections between PE_0 1104 and SB_1 1110 and between SB_1 1110and PE_1 1106 may be shown in solid lines to indicate data may passthrough. A double-lined arrow 1118 may illustrate this data flow.

Because the output from PE_0 1104 may only be needed by PE_1 1106 in thefirst PDP, at this moment, no data may need to pass through ICSB_1 1114.Thus, although ICSB_1 1114's configuration may be programmed already(e.g., it's internal connection shown in a dash dotted line), but thereis no data coming to ICSB_1 1114 (e.g., its connection to SB_1 1110 in adotted line) and ICSB_1 1114 may stay still.

At the stage shown in FIG. 11D, the configuration connection from SEQ1102 to PE_1 1106 may be open and illustrated as dashed lines. Incontrast, the other configuration connections may be closed and shown indotted lines. SEQ 1102 may now configure a second instruction INS2 toPE_1 1106, and PE_1 1106 may be ready to execute and generate outputsbut blocked by the output busy signal from SB_2 1112. At the same time,the data generated from PE_0 1104 may continuously be sent to D-FIFO ofPE_1 1106. At this stage, all data connections between the componentsmay be the same as in FIG. 11C.

In FIG. 11E, at this stage, the configuration connection from SEQ 1102to SB_2 1112 and ICSB 1116 may be open and illustrated as dashed lines.In contrast, the other configuration connections may be closed and shownin dotted lines. When SB_2 1112 may be configured and open, PE_1 1106may start executing INS2 and generate outputs. These outputs may berouted via SB_2 1112 (e.g., a dash dotted line inside the SB_2 1112 toshow internal connection open) to PE_2 1108. PE_2 1108 may receive theoutputs from PE_1 1106 and store these data signals in its data buffer(e.g., D-FIFOs) even when PE_2 1108 has not been configured yet. Thedata connections between PE_1 1106 and SB_2 1112 and between SB_2 1112and PE_2 1108 may be shown in solid lines to indicate data may passthrough. A double-lined arrow 1120 may illustrate this data flow.

Because in the first PDP, the input to PE_2 1108 may only come from PE_11106, at this moment, no data may need to pass through ICSB_2 1116.Thus, although ICSB_1 1116's configuration may be programmed already(e.g., it's internal connection shown in a dash dotted line), but thereis no data passing through ICSB_2 1116 (e.g., its connection to SB_21112 shown in a dotted line) and ICSB_2 1116 may stay still.

At the stage shown in FIG. 11F, the configuration connection from SEQ1102 to PE_2 1108 may be open and illustrated as dashed lines. Incontrast, the other configuration connections may be closed and shown indotted lines. And the last instruction INS3 of the first PDP may beprogrammed to PE_2 1108. Although not shown, at this stage, theconfiguration connection to a gasket memory (e.g., an embodiment of thegasket memory 216) may also be opened, and the data connection of thegasket memory may also be programmed. Now PE_2 1108 may execute INS3 andresult may be stored into the gasket memory. In one embodiment, by now,the entire PDP_1 configuration may be completed and each PE (e.g., PE_01104, PE_1 1106 and PE_2 1108) may independently execute the respectiveinstructions a specified number (e.g., NUM_EXEC) of times. Each SB(e.g., SB_1 1110 and SB_2 1112) and each ICSB (e.g., ICSB_1 1114 andICSB_2 1116) may also execute their respective configurations thespecified number (e.g., NUM_EXEC) times for PDP_1.

In some embodiments, configurations for PDPs of a VDP (e.g., for adependency graph of an execution kernel) may be sent to the componentsindependently while each component may be operating according to acurrent configuration. For example, while the PEs (e.g., PE_0 1104, PE_11106 and PE_2 1108), SBs (e.g., SB_1 1110 and SB_2 1112) and ICSBs(e.g., ICSB_1 1114 and ICSB_2 1116) may be operating under theirrespective first configuration for PDP_1, subsequent configurations ofother PDPs of the same VDP for each of these components may be receivedfrom the SEQ 1102. In one embodiment, a plurality of configurations forone component may be sent via the configuration bus from a sequencer 206in a batch as long as sending multiple configurations for one componentwill not slow down or block the operation of any other components.

Therefore, while PDP_1 may be carried out, all the configurations forPDP_2 may have been received by the components. As shown in FIG. 11G, atthis stage, by the time when all threads (e.g., all ALUs in an ALUvector 602) on PE_0 1104 have completed PDP_1, PE_0 1104 may bereconfigured to a fourth instruction INS4. For example, PE_0 1104 mayreconfigure by itself by applying the PDP_2 configuration already in itsconfiguration buffer C-FIFO 614. Similarly, SB_1 110 may also bereconfigured once the last data from INS1 has passed it. The firstoutput data from INS4 may be delivered to ICSB_1 1114 due to the newPDP_2 configuration on SB_1 1110 (e.g., a new dash dotted lineconnecting to ICSB_1 1114). The data signal to ICSB_1 1114 may beblocked, however, by a busy signal from ICSB_2 1116 because ICSB_2 1116may have not been reconfigured yet.

At the stage shown in FIG. 11H the PE_1 1106, SB_2 1112, ICSB_2 1116 mayhave reached the specified number of executions in PDP_1, and may bereconfigured. For example, PE_1 1106 may be reconfigured to a fifthinstruction INS5, and SB_2 1112 and ICSB_2 1116 may also be reconfiguredso that the data from PE_0 1104 may also reach the D-FIFO of PE_2 1108via ICSB_1 1114. A new dash dotted line inside SB_2 1112 may be shown toconnect ICSB_2 1116 to PE_2 1108 via SB_2 1112. The data flow from PE_01104 to PE_2 1108 may be illustrated by the double-lined arrows 1122(e.g., from PE_0 1104 to ICSB_1 1114) and 1124 (e.g., from ICSB_1 1114to PE_2 1108).

At the stage shown in FIG. 111 at last, PE_2 1108 may reach thespecified number of execution for PDP_1 and may be reconfigured to alast instruction INS6 for PDP_2. The entire PDP_2 configuration may beapplied to the data path components. The components for PDP_2configuration may each run a specified number of times for the PDP_2configuration to complete the entire execution kernel.

FIG. 12A schematically shows a pipeline operation for executing a streamof instructions using a processor in accordance with an embodiment ofthe present disclosure. In a shared memory access mode, one memoryaddress may be accessed by all threads executing same instruction. Toload data from the same address for all the threads, the pipeline maykeep on fetching the same data for all these threads and createsunnecessary traffic in the memory unit. The instructions in this examplemay be denoted with an underline in front of a numeral to distinguishthe example instructions shown in FIGS. 11A-11I. In the examplepipelined instruction execution shown in FIG. 12A, instruction Ins_0 maybe a data loading instruction “Load x[j]” and a memory port may beconfigured to execute Ins_0 three times (e.g., NUM_EXEC being 3 for amemory port configured to execute the load instruction) as a pipelinestage 1202. The data piece x[j] may be common to all threads and loadedfrom the same address. For example, the data piece x[j] may be a j-thdata piece in a vector x, and this j-th data piece may be used by allthreads. In an embodiment with N ALUs in one PE, there may be Nconcurrent threads represented by one block (e.g., one block of threads)and the pipeline stage 1202 may be configured to execute Ins_0 3×Ntimes.

Instruction Ins_1 may be a data loading instruction “Load a[k][j]” and amemory port may be configured to execute Ins_1 three times as a pipelinestage 1204. The data piece to be loaded by Ins_1 may be different fordifferent threads and may be loaded from different addresses fordifferent threads. For example, a[k][j] may be a j-th data piece for ak-th thread, with k may be an integer between 0 to N−1 (inclusive) foreach thread in the first block of threads, between N to 2N−1 (inclusive)for each thread in the second block of threads, and between 2N to 3N−1(inclusive) for each thread in the third block of threads.

In one embodiment, the pipeline stages 1202 and 1204 may be performed ata same memory port if the memory port is configured to carry out twodata loading instructions in parallel. For example, two parallel readdata lines and two parallel write data lines between each of the MPs 220and the memory unit 212 are shown in FIG. 2. Moreover, the switch box500 shows how the parallel input data lines may be switched to a PE(e.g., 502.1 and 502.2 switched to 506.1, 506.2, 508.1 or 508.2). Inanother embodiment, the pipeline stages 1202 and 1204 may be performedat two different memory ports.

Instruction Ins_2 may be a multiplication instruction “y=a[k][j]*x[j]”with the data piece x[j] being loaded by Ins_0 and a[k][j] being loadedby Ins_1 and a PE may be configured to execute Ins_2 three times (e.g.,NUM_EXEC being 3 with totally 3×N times for all threads) as a pipelinestage 1206. Therefore, each PE or MP may be configured to executeNUM_EXEC amount of instructions as a pipeline stage.

Instruction Ins_4 may be a data loading instruction “Load x[j+1]” and amemory port may be configured to execute Ins_4 three times as a pipelinestage 1208. The data piece x[j+1] may be common to all threads andloaded from the same address. For example, the data piece x[j+1] may bea j+1-th data piece in the vector x, and this j+1-th data piece may beused by all threads. Instruction Ins_5 may be a data loading instruction“Load a[k][j+1]” and a memory port may be configured to execute Ins_5three times as a pipeline stage 1210. The data piece to be load by Ins_5may be different for different threads and may be loaded from differentaddresses for different threads. For example, a[k][j+1] may be a j+1-thdata piece for a k-th thread, with k may be an integer between 0 to N−1(inclusive) for each thread in the first block of threads, between N to2N−1 (inclusive) for each thread in the second block of threads, andbetween 2N to 3N−1 (inclusive) for each thread in the third block ofthreads. In one embodiment, the pipeline stages 1208 and 1210 may beperformed at a same memory port if the memory port is configured tocarry out two data loading instructions in parallel. In anotherembodiment, the pipeline stages 1208 and 1210 may be performed at twodifferent memory ports.

Instruction Ins_6 may be a multiplication instruction“y=a[k][j+1]*x[j+1]” with the data piece x[j+1] being loaded by Ins_4and a[k][j+1] being loaded by Ins_5 and a PE may be configured toexecute Ins_6 three times as a pipeline stage 1212.

In the example pipelined instruction execution of FIG. 12A, Ins_0 andIns_4 may be repeated although many repeated memory reads may not benecessary. FIG. 12B schematically shows a reduced pipeline operation forexecuting a stream of instructions using a processor in accordance withan embodiment of the present disclosure. In one embodiment, a compileror other method (e.g., the sequencer) may identify the static read inthe sequence of instructions in FIG. 12A and the repeated operations maybe reduced to one time. For example, the static load instruction (e.g.,Ins_0 and Ins_4) may be programmed with NUM_EXEC=1 (e.g., this may beapplied to the MP that will perform data fetching). As shown in FIG.12B, the pipelined stages 1202A and 1208A may be one blocksrespectively. In one embodiment, indicators S may be implemented in thePEs and MPs next to PEs' or MPs' D-FIFO (e.g., indicators 622 in a PE620), and for any PEs and MPs that that receive data from the staticload, the indicator S next to PEs' or MPs' D-FIFO that receive data fromthe static load may be set. For example, the PE configured to executeIns_2 at pipeline stage 1206 may comprise D-FIFOs that receive data fromthe MP(s) configured for pipeline stages 1202A and MP 1204 respectively,and an indicator S may be set next to the D-FIFO that receives data frompipeline stage 1202A. Similarly, the PE configured to execute Ins_6 atpipeline stage 1212 may comprise D-FIFOs that receive data from theMP(s) configured for pipeline stages 1208A and 1210 respectively, and anindicator S may be set next to the D-FIFO that receives data from thepipeline stage 1208A. With the NUM_EXEC=1, the data load operation inpipeline stages 1202A and 1208A may be executed only once for thisconfiguration. Given the static indication S, the PEs configured toperform pipeline stages 1206 and 1212 may execute its operation threetimes (e.g., their NUM_EXEC is still equal to 3) but data from theD-FIFO with indication S may be reused for NUM_EXEC times.

In some embodiments, this operation mode with reduced pipeline stage maybe generalized to other instructions. In one embodiment, for aninstruction that may generate the same result for different threads, thesame approach can be used to reduce power consumptions. For example, aresult from one PE may be used as an input for different threads inanother PE in the same physical data path, or a result from a PE of onephysical data path may be used as an input for different threads in a PEin another physical data path, the result may be loaded only once withthe indication S set for a corresponding D-FIFO and reused.

Referring to FIG. 13, a flowchart of a process 1300 for executing anexecution kernel is depicted in accordance with an embodiment of thepresent disclosure. At block 1302, an execution kernel may be mappedinto a virtual data path at a processor. For example, as shown in FIG.9B, an execution kernel may be mapped into a virtual data path by anexample processor 200. The execution kernel may include a sequence ofinstructions to be executed by the processor. In one embodiment, theprocessor may comprise various reconfigurable units that include agasket memory. Moreover, in one embodiment, the processor may comprisevarious reconfigurable units that form repetitive columns. At block1304, the virtual data path may be chopped into one or more physicaldata paths. For example, as shown in FIG. 9C, a virtual data path may bechopped into three physical data paths. In one embodiment, the variousreconfigurable units may form the one or more physical data paths toexecute the sequence of instructions. Moreover, in one embodiment, eachof the one or more physical data paths may be fitted into the repetitivecolumns respectively. For example, a first physical data path and asecond physical data path may be fitted into the repetitive columnsrespectively. At block 1306, configurations may be delivered to variousreconfigurable units of the processor. The various reconfigurable unitsmay form the one or more physical data paths according to theconfigurations to execute the sequence of instructions. At block 1308,the processor may be executed to complete the one or more physical datapaths by operating the various reconfigurable units according to theconfigurations. In one embodiment, data from one physical data path maybe routed to the gasket memory to be used in a future physical data pathas input.

Referring to FIG. 14, a flowchart of a process 1400 for reconfiguring aprocessor is depicted in accordance with an embodiment of the presentdisclosure. At block 1402, a plurality of configurations may bedelivered to a plurality of reconfigurable units of a processor. Theplurality of configurations may be for the plurality of reconfigurableunits to form a plurality of physical data paths to execute a sequenceof instructions. In one embodiment, each of the plurality ofconfigurations may include a specified number (e.g., the NUM_EXEC numberas described herein). At block 1404, a respective operation may berepeated at each of the plurality of reconfigurable units for aspecified number of times. For example, the plurality of reconfigurableunits may include a first reconfigurable processing element (PE), andthe first reconfigurable PE may execute a first instruction of thesequence of instructions the specified number of times in a firstphysical data path of the plurality of physical data paths. At block1406, each reconfigurable unit may be reconfigured to a newconfiguration. In one embodiment, after each reconfigurable unit havingrepeated its respective operation the specified number of times, eachreconfigurable unit may be reconfigured. For example, the firstreconfigurable PE may be reconfigured to execute a second instruction ofthe sequence of instructions the specified number of times according toa second configuration in a second physical data path of the pluralityof physical data paths.

Referring to FIG. 15, a flowchart of a process 1500 for reconfiguring aprocessor is depicted in accordance with an embodiment of the presentdisclosure. At block 1502 a first configuration and a secondconfiguration may be received at a reconfigurable unit of a processor.For example, PE_0 1104 in FIGS. 11A-11I may receive a firstconfiguration that may comprise the first instruction INS1 and a secondconfiguration that may comprise the fourth instruction INS4. PE_0 1104may store these configurations in its configuration buffer (e.g., C-FIFO614). At block 1504, a first operation may be executed a first number oftimes according to the first configuration. For example, PE_0 1104 mayexecute the instruction INS1 a number of times according to the NUM_EXECin the first configuration. The first configuration may be part of afirst physical data path (e.g., PDP_1) for executing a first part of asequence of instructions. At block 1506, the reconfigurable unit may bereconfigured to execute a second operation a second number of timesaccording to the second configuration. For example, PE_0 1104 may bereconfigured by applying the second configuration to execute theinstruction INS4 a number of times according to the NUM_EXEC in thesecond configuration. The second configuration may be part of a secondphysical data path (e.g., PDP_2) to execute a second part of thesequence of instructions.

Referring to FIG. 16, a flowchart of process 1600 for reconfiguring aprocessor is depicted in accordance with an embodiment of the presentdisclosure. At block 1602, a first instruction may be executed at areconfigurable processing element a number of times. The execution maybe performed according to a first configuration as part of a firstphysical data path. For example, PE_2 may be configured to execute INS3NUM_EXEC times according to a configuration as part of PDP_1. At block1604, an execution result from the reconfigurable unit may be deliveredto a gasket memory. For example, the execution result from PE_2 forexecution of INS3 may be delivered to a gasket memory. In oneembodiment, if the first instruction is to be executed a number of timesat the reconfigurable unit, the execution result may be delivered to thegasket memory after each execution of the first instruction. Forexample, the gasket memory may store the successive execution results inone of its data buffers (e.g., one of D-FIFOs 802.3 through 802.F). Atblock 1606, the execution result stored in the gasket memory may be fedfrom the gasket memory to a second physical data path. For example, theexecution result from PE_2 executing the instruction INS3 may be used asinput for an instruction to be executed by a PE in the second physicaldata path. The execution result stored in the gasket memory may then bedelivered from the gasket memory to that PE for executing the secondphysical data path.

Referring to FIG. 17, a flowchart of a process 1700 for accessing memoryis depicted in accordance with an embodiment of the present disclosure.At block 1702, a plurality of memory addresses for a memory unit may begenerated by an address calculation unit in a memory port for parallelthreads. For example, the memory port may provide memory access to amemory unit for a processing element (PE) that may have a plurality ofarithmetic logic units (ALUs) configured to execute a same instructionin parallel threads. In one embodiment, the memory port may be one of aplurality of memory ports that provide access to a memory unit for aplurality of processing elements (PEs). Each of the plurality of PEs mayhave a plurality of arithmetic logic units (ALUs) configured to executea same instruction in parallel threads. At block 1704, a plurality ofmemory banks in the memory unit may be accessed with each threadaccessing a different memory bank. In one embodiment, in a privatememory access mode, each thread may access a different memory bank forits data.

Referring to FIG. 18, a flowchart of a process 1800 for accessing memoryis depicted in accordance with an embodiment of the present disclosure.At block 1802, a plurality of memory addresses for a memory unit may begenerated by an address calculation unit in a memory port for parallelthreads. The memory port may provide memory access for a processingelement that has a plurality of arithmetic logic units (ALUs) configuredto execute a same instruction in parallel threads. In one embodiment,the memory port may be one of a plurality of memory ports that provideaccess to a memory unit for a plurality of processing elements (PEs)each having a plurality of arithmetic logic units (ALUs) that areconfigured to execute a same instruction in parallel threads. At block1804, a plurality of memory banks in the memory unit may be accessedwith all threads accessing a common area in the memory unit. In oneembodiment, in a shared memory access mode, each of the parallel threadsexecuting by a PE may have access anywhere in the common area. That is,the common area may be a shared memory space for all threads.

Referring to FIG. 19, a flowchart of a process 1900 for reusing a pieceof data is depicted in accordance with an embodiment of the presentdisclosure. At block 1902, it may be determined that a piece of data isto be shared and reused by all threads at a processing element (PE) of aprocessor during one configuration applied at the PE. For example, inone embodiment, a shared memory access mode may be applied during oneconfiguration for a PE. A piece of data (e.g., a constant) may be commonto all threads, and may be shared and reused by all threads. At block1904, the piece of data may be loaded once into a data buffer of the PE.In an embodiment, because the piece of data may be shared and reused,the data load operation may need to be performed only once. At block1906, an indicator associated with the data buffer may be set toindicate that the piece of data is to be reused. In one embodiment, thedata buffer inside the PE may have indicator bit “S” and this indicator“S” may be set if a piece of data in the data buffer should be reused.At block 1908, a same instruction may be executed a number of times byreusing the piece of data as an input. In one embodiment, the PE mayrepeatedly execute the same instruction as a pipeline stage according toa number (e.g., NUM_EXEC) specified by the configuration.

The present disclosure provides apparatus, systems and methods forreconfigurable parallel processing. For example, an embodiment of a RPPmay utilize a 1-D or 2-D data path which consists of a processingelement (PE) array and interconnections to process massive paralleldata. The data path may be made identical in each section (e.g., onecolumn of PE(s), MP and data routing units), which may allow thedependency graph of a kernel to be mapped to a virtual data path thatmay be an infinite repetition of the data path sections in onedimension.

An embodiment of a RPP may also utilize a gasket memory to temporallystore data output of data paths as where the virtual data path issegmented into physical data paths. The gasket memory may function likea data buffer (e.g., FIFO) to feed data back into physical data path ofthe next configuration.

An embodiment of a RPP may also have a one-dimensional memory unit withmemory ports (MPs) connected to each column of data path. All dataaccessed throughout the virtual data path may be stored in the memoryunit. Each time for a new configuration, a MP may be reconfigured toaccess the memory unit differently while the data could stay the same.

An embodiment of a RPP may separate types of memory access to privatememory access and shared memory access. Private memory access may bededicated to a particular thread with no overlapping access allowedbetween different threads. Shared memory access may allow all threads toaccess common area. Instead of defining different memories for sharedand private types. An embodiment of a RPP may store data into the samememory space but provides different access methods. This eliminatesunnecessary data movement from private memory to shared memory and viceversa.

Embodiments of a RPP may be optimized to allow massive parallelism formultithread processing. In one example, with one row of 32 PEs and eachPE having 32 arithmetic and logic units (ALUs), 1024 ALUs may beincluded in one RPP core. In some embodiments, a multi-core processormay comprise multiple RPPs.

Embodiments of a RPP may be reconfigured according to a reconfigurationmechanism. The various components of a RPP that include one or morereconfiguration counters may be referred to as reconfigurable units. Forexample, each of the PEs (e.g., PE 218), the switching units (e.g., SB222 and ICSB 224) and memory units (e.g., MP 220, gasket memory 216),may comprise one or more reconfiguration counters, such as the counter606 in a PE, the counters 520 in a SB, the counters 714 in an ICSB, thecounters 820 and 822 in a gasket memory, and similar counters in a MP(not shown in FIG. 4A or 4B). Data processing may be pipelined whenthere may be no dependency between threads. Identical instructions maybe executed multiple times until all threads (e.g., four blocks ofthreads for one physical data path) are processed. When counters in areconfigurable unit reach a programmed number, the reconfigurable unitmay replace its configuration to a new context. This reconfiguration maybe done in the same way in each PE, switch boxes and memory accessunits. Self-reconfiguration may be achieved with minimum idle time forswitching.

The exemplary reconfiguration mechanism may reduce the power spent onconfiguration because the configuration is only switched once after allthreads have been processed. This may also reduce idle time betweenconfigurations by switching each PE independently at its earliest time.By doing that, the memory required to store intermediated data may alsobe reduced.

In some embodiments, all threads may load data use the same address in ashared memory access mode. Due to the pipelined nature of operation,only the first data load instruction of all threads may need to beperformed. The data loaded may be shared with all threads to reduce thememory access traffic and power consumption.

The techniques described herein may be implemented in one or moreapplication specific integrated circuits (ASICs) in digital logic gates,or by a processor that execute instructions stored in a tangibleprocessor readable memory storage media.

In one embodiment, any of the disclosed methods and operations may beimplemented in software comprising computer-executable instructionsstored on one or more computer-readable storage media. The one or morecomputer-readable storage media may include non-transitorycomputer-readable media (such as removable or non-removable magneticdisks, magnetic tapes or cassettes, solid state drives (SSDs), hybridhard drives, CD-ROMs, CD-RWs, DVDs, or any other tangible storagemedium), volatile memory components (such as DRAM or SRAM), ornonvolatile memory components (such as hard drives)). Thecomputer-executable instructions may be executed on a processor (e.g., amicrocontroller, a microprocessor, a digital signal processor, etc.).Moreover, an embodiment of the present disclosure may be used as ageneral-purpose processor, a graphics processor, a microcontroller, amicroprocessor, or a digital signal processor.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

What is claimed is:
 1. A processor, comprising: a plurality ofprocessing elements (PEs) each comprising a configuration buffer; amemory unit for providing data storage for the plurality of PEs; aplurality of memory ports chained to form a data routing bus and coupledto the plurality of PEs for the plurality of PEs to access the memoryunit; a sequencer coupled to the configuration buffer of each of theplurality of PEs and configured to distribute one or more individual PEconfigurations to each respective PE of the plurality of PEs; and agasket memory coupled to the plurality of PEs and configured to store atleast one PE execution result from any of the plurality of PEs to beused by at least one of the plurality of PEs during a next PEconfiguration.
 2. The processor of claim 1, further comprising: aplurality of switch boxes each associated with one of the plurality ofPEs for the plurality of switch boxes to collectively provide dataswitching for the plurality of PEs; and wherein the plurality of PEs arearranged in a plurality of columns of PEs and the plurality of switchboxes are a plurality of columns of switch boxes and each of theplurality of memory ports is coupled to a separate column of theplurality of columns of switch boxes for the plurality of PEs to accessthe memory unit via the plurality of switch boxes.
 3. The processor ofclaim 2, further comprising a plurality of inter-column switch boxes(ICSBs) coupled to the sequencer to receive ICSB configurations from thesequencer, the plurality of ICSBs configured to provide data switchingbetween neighboring columns of the plurality of columns of switch boxesaccording to the ICSB configurations.
 4. The processor of claim 2,wherein the plurality of memory ports (MPs) are coupled to the sequencerto receive MP configurations from the sequencer and configured tooperate in a private memory access mode or a shared memory access modeduring one MP configuration.
 5. The processor of claim 4, wherein apiece of data stored in the memory unit is accessed at a same storagelocation through the private memory access mode and the shared memoryaccess mode from different part of a program.
 6. The processor of claim2, wherein each of the plurality of columns of PEs comprises one PE, andwherein the plurality of PEs are identical and chained using theplurality of switch boxes to form one row of identical PEs.
 7. Theprocessor of claim 2, wherein each of the plurality of columns of PEscomprises two or more PEs and the plurality of PEs form two or more rowsof PEs.
 8. The processor of claim 7, wherein each PE of a first row ofPEs is configured to implement a first set of instructions and each PEof a second row of PEs is configured to implement a second set ofinstructions, and at least one instruction of the second set ofinstructions is not in the first set of instructions, wherein theplurality of columns of PEs are identical columns.
 9. The processor ofclaim 1, wherein each of the plurality of PEs comprises a plurality ofarithmetic logic units (ALUs) and the plurality of ALUs of one PE areconfigured to execute a same instruction in parallel threads.
 10. Theprocessor of claim 9, wherein each of the plurality of PEs comprises aplurality of data buffers for the plurality of ALUs and is configured tooperate independently.
 11. A system, comprising: a processor,comprising: a sequencer configured to map an execution kernel to beexecuted by the processor into a virtual data path and chop the virtualdata path into two or more physical data paths; a plurality ofprocessing elements (PEs) coupled to the sequencer, each of theplurality of PEs comprising a configuration buffer configured to receivePE configurations for the two or more physical data paths from thesequencer; a memory unit for providing data storage for the plurality ofPEs; a plurality of memory ports chained to form a data routing bus andcoupled to the plurality of PEs for the plurality of PEs to access thememory unit; and a gasket memory coupled to the plurality of PEs andconfigured to store data from one of the two or more physical data pathsto be used by another physical data path of the two or more physicaldata paths as input.
 12. The system of claim 11, wherein the processorfurther comprises a plurality of switch boxes (SBs) coupled to thesequencer to receive SB configurations for two or more physical datapaths from the sequencer, each of the plurality of switch boxesassociated with a respective PE of the plurality of PEs and configuredto provide data switching for the respective PE according to the SBconfigurations.
 13. The system of claim 11, wherein the plurality ofmemory ports (MPs) are coupled to the sequencer to receive MPconfigurations for the two or more physical data paths from thesequencer and configured to operate in a private memory access mode or ashared memory access mode during one MP configuration.
 14. The system ofclaim 13, wherein a piece of data stored in the memory unit is accessedat a same storage location through the private memory access mode andthe shared memory access mode from different physical data paths of thetwo or more physical data paths.